acceptodds
Under review as a conference paper at ICLR 2027

Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork

Abstract

Differentially private (DP) training of neural networks is often hindered by the large amount of noise required by gradient-based methods such as DP-SGD, which repeatedly inject high-dimensional noise in parameter space throughout training. In this paper, we propose a new framework for DP learning that avoids iterative optimization in parameter space. Instead of updating the target model using privatized gradients, we employ a *hypernetwork trained on public datasets* to map a private dataset to the parameters of the target model. Specifically, each example is embedded into a low-dimensional representation, the embeddings are aggregated and perturbed to obtain a DP dataset embedding, and the hypernetwork generates the target model parameters from this noisy embedding. Because privacy noise is injected only once into a low-dimensional dataset representation, our approach can significantly reduce the adverse effect of noise. We theoretically show in a synthetic setting that, under a fixed privacy budget, models produced by our approach achieve higher utility than those trained with DP-SGD. Moreover, we apply our approach to LoRA fine-tuning of diffusion models and show that it achieves lower FID than LoRA models trained with DP-SGD and other public-data-guided methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
differential privacy, hypernetwork

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_FtcgOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DP-DeepSets, a hypernetwork trained on public data to generate model parameters from private datasets. Instead of repeatedly adding noise to gradients as in DP-SGD, the method adds noise once to an aggregated dataset embedding and uses it to generate model parameters.

Critical strengths

Using a public hypernetwork to generate private model parameters is a promising way to avoid repeated noise injection during DP training. The theoretical analysis also provides some insight into why learning low-dimensional task representations could help.

Critical weaknesses

1. The experimental evaluation is very limited.

I do not understand why diffusion-model fine-tuning was chosen as the only evaluation task. The paper proposes a general DP learning method, but evaluates only one task and one target-model architecture. Why not test on standard DP benchmarks such as CIFAR-10 classification? This would make it easier to compare against existing DP methods and understand the actual improvement.

2. The baselines are not sufficiently strong.

The paper mainly compares against relatively basic DP optimization methods. Stronger DP-SGD training recipes, such as those in Unlocking High-Accuracy Differentially Private Image Classification through Scale (De et al., 2022, https://arxiv.org/abs/2204.13650), should be considered on a suitable classification benchmark.

I am also concerned about the baseline hyperparameters in Appendix E. For example, Table 2 reports unusually large learning rates for DP-SGD. The authors should clarify these settings and demonstrate that the baselines are properly tuned.

3. Important reference results are missing.

The paper should report the performance of:

a. The original pretrained model without fine-tuning.

b. Non-private fine-tuning on the same private datasets.

c. Fine-tuning using only public data, without private adaptation.

Without these results, it is difficult to know how good the proposed method actually is, how much utility is lost due to privacy, and how much improvement comes from private data rather than public training.

4. The privacy budgets are questionable.

The experiments include privacy budgets as large as ϵ=210=1024\epsilon=2^{10}=1024. Such values provide little meaningful privacy protection. More concerningly, DP-SGD still performs poorly at these extremely large budgets. This raises questions about whether the baseline is properly optimized. The evaluation should focus more on meaningful privacy budgets and include non-private reference results.

5. The scope of the privacy guarantee is unclear.

The privacy proof considers a fixed input dataset, while the experiments construct overlapping private datasets using k-NN and generate multiple models. It is unclear whether the reported privacy guarantee applies to individual 128-example tasks or to the entire data-processing and model-release procedure.

Questions for authors
  1. Why diffusion models? Can the authors evaluate the method on standard classification tasks such as CIFAR-10, preferably with more than one model architecture?

  2. Can the authors provide stronger baselines? In particular, how do the results compare with carefully tuned DP-SGD methods? Are the unusually large learning rates in Appendix E correct?

  3. Can the authors report non-private and public-only results? These are essential for understanding the actual privacy-utility trade-off. How much does conditioning on the private dataset improve over using public data alone?

  4. Why evaluate at ϵ=1024\epsilon=1024? Is this intended to approximate non-private training? If so, why does DP-SGD still perform poorly?

  5. What is the exact privacy guarantee? Does it cover the entire experimental pipeline, including k-NN task construction and multiple model releases, or only each fixed-size private task?

6 comments
AuthorsOpenReview ↗

Response to Reviewer Ftcg (1/2)

We thank the reviewer for the careful reading and the concrete suggestions. We address the concerns and questions below; all revisions mentioned here are included in the revised PDF.

W1 & Q1: Classification experiments and the choice of diffusion models

I do not understand why diffusion-model fine-tuning was chosen as the only evaluation task. ... Why not test on standard DP benchmarks such as CIFAR-10 classification? ...

Can the authors evaluate the method on standard classification tasks such as CIFAR-10, preferably with more than one model architecture?

New classification experiments. Following the suggestion, we have added classification on CIFAR-10 (binary) and CIFAR-100 (10-way) to Section 5. Since our target is adapting a pretrained model to a small private dataset, each private task again has 128 examples, and private tasks use class pairs (CIFAR-10) or classes (CIFAR-100) unseen during public training. This differs from the standard benchmark that uses the full CIFAR-10 training set, so the accuracies are not comparable to those reported there. With only 128 examples, DP-SGD needs far more noise relative to the gradient signal than in the full-dataset setting. At ϵ=1\epsilon=1 and δ=10−5\delta=10^{-5}, accuracy over 100 private tasks is 98.4% versus 78.5% for DP-SGD on CIFAR-10, and 34.3% versus 12.3% on CIFAR-100. Our method is better at all six budgets from ϵ=0.25\epsilon=0.25 to 88. The full table is in Section 5 and in our response to Reviewer n6F4.

More than one architecture. The hypernetwork generates a linear classifier on frozen CLIP features, so the target models now include both the LoRA of a diffusion U-Net and a linear classifier.

Why diffusion models. We chose diffusion-model fine-tuning because generating low-rank weights of a text-to-image diffusion model from reference images is a known use of hypernetworks (Ruiz et al., 2024).

W2 & Q2: Stronger baselines and the learning rates

Stronger DP-SGD training recipes, such as those in ... (De et al., 2022), should be considered on a suitable classification benchmark. ... The authors should clarify these settings and demonstrate that the baselines are properly tuned.

Are the unusually large learning rates in Appendix E correct?

Stronger and re-tuned DP-SGD. In both tasks, we have re-tuned DP-SGD on public tasks only, including large batches up to the full 128 examples as recommended by De et al. (2022), together with the learning rate, the number of updates, and the clipping threshold. In classification, where DP-SGD trains the same linear head on the same frozen features, the gap remains large, e.g., 52.8% versus 15.8% on CIFAR-100 at ϵ=2\epsilon=2. In generation, the re-tuned FIDs at ϵ=1\epsilon=1 and 44 are 369.0 and 257.0, while ours are 137.4 and 133.4, so our method remains better by more than 120 in FID. Appendix E gives the details.

Learning rates of 10210^2 and 10310^3. Yes, the values are correct. Each DP-SGD update has norm at most the learning rate times the clipping threshold CC, apart from noise, and the per-example gradients of rank-1 LoRA are small (median norm about 1.7×10−31.7\times10^{-3} on public tasks). A learning rate of 10310^3 therefore gives updates of moderate size. DP-AdamW, which normalizes the gradient scale, selects ordinary rates of 0.003–0.01.

AuthorsOpenReview ↗

Response to Reviewer Ftcg (2/2)

W3 & Q3: Reference results

a. The original pretrained model without fine-tuning. b. Non-private fine-tuning on the same private datasets. c. Fine-tuning using only public data, without private adaptation.

We have added all three, evaluated on the same 100 private tasks with 1,024 generated images each. (a) is the ImageNet64 checkpoint without LoRA. (b) trains the same rank-1 LoRA with AdamW on each private task without clipping or noise, with its learning rate and number of updates selected on public tasks. (c) is the model fine-tuned on all public examples, which the Public FT + DP-SGD baseline starts from, without private adaptation. FID (mean ±\pm SD):

ModelFID
(a) Pretrained model251.4 ±\pm 19.6
(c) Public-only fine-tuning230.4 ±\pm 29.1
Ours, ϵ=1\epsilon=1137.4 ±\pm 24.3
Ours, ϵ=4\epsilon=4133.4 ±\pm 23.2
(b) Non-private fine-tuning106.6 ±\pm 21.7

Public-only fine-tuning lowers the FID of the pretrained model by only 21, whereas conditioning on the private dataset lowers it by a further 93 at ϵ=1\epsilon=1, which closes 75% of the gap to non-private fine-tuning. At ϵ=1\epsilon=1, the FID of our method is 30.8 higher than that of non-private fine-tuning, whereas that of DP-AdamW, which uses the same optimizer, is 144.5 higher (FID 251.1). Section 5 also reports non-private results for classification.

W4 & Q4: Privacy budgets and ϵ=1024\epsilon=1024

The experiments include privacy budgets as large as ϵ=210=1024\epsilon = 2^{10} = 1024. Such values provide little meaningful privacy protection. More concerningly, DP-SGD still performs poorly at these extremely large budgets.

Is this intended to approximate non-private training? If so, why does DP-SGD still perform poorly?

Figure 2 sweeps ϵ\epsilon from 0.125 to 1024 to show how each method behaves as the privacy constraint is relaxed, and ϵ=1024\epsilon=1024 is the end of this sweep. It is not meant to approximate non-private training, and our main claims concern ϵ≤8\epsilon\le8, as in Table 1 (ϵ∈{1,4}\epsilon\in\lbrace1,4\rbrace) and the new classification experiments (ϵ\epsilon from 0.25 to 8). As the non-private reference, we have added fine-tuning without clipping or noise (W3).

We agree that the submitted DP-SGD was not tuned well enough. In particular, its C=0.01C=0.01 was too large for gradients of median norm about 1.7×10−31.7\times10^{-3}. Re-tuning on public tasks reduced CC to 1.7×10−31.7\times10^{-3} and the FID at ϵ=1024\epsilon=1024 from 215.3 to 152.1, close to DP-AdamW with a tuned CC (153.5). Even at ϵ=1024\epsilon=1024, the 800 updates on only 128 examples require a noise multiplier of 0.69. Since the noise is added to all 40,704 LoRA parameters, its norm in each averaged update is about 1.1C1.1C, larger than that of the averaged clipped gradient (at most CC). Our method at every ϵ\epsilon from 1 to 128 (FID between 132.8 and 137.4) is better than DP-SGD at ϵ=1024\epsilon=1024.

W5 & Q5: Scope of the privacy guarantee

It is unclear whether the reported privacy guarantee applies to individual 128-example tasks or to the entire data-processing and model-release procedure.

The guarantee applies to each 128-example task. Our mechanism takes one fixed task as input and outputs one model, and its (ϵ,δ)(\epsilon,\delta)-DP guarantee protects each example of that task. It does not cover the kk-NN construction and the release of all 100 models jointly. The kk-NN construction is only our way of creating many small private datasets for evaluation, as in a setting where each data owner holds one small dataset, and all baselines are given the same per-task guarantee. If the private pool were instead regarded as a single dataset, an image used in several tasks would incur composition over those releases, and the construction itself would also have to be private. We do not claim such a guarantee, and we have stated the scope explicitly in Section 5.


We would be happy to clarify any remaining concerns during the discussion period. We would greatly appreciate it if you could consider increasing the score once all concerns have been resolved.

AuthorsOpenReview ↗

Further revised PDF and summary of revisions

Thank you again for your review. We have uploaded a further revised PDF in which DP-SGD in Figure 2 is re-tuned at all budgets. Our method remains better at every budget from ϵ=1\epsilon=1 to 128128, where both are evaluated, e.g., FID 132.8 versus 251.9 at ϵ=8\epsilon=8. At large budgets, the re-tuned DP-SGD is now close to DP-AdamW, e.g., 152.1 versus 153.5 at ϵ=1024\epsilon=1024.

The main revisions in response to your review are as follows, with details in our response above.

  • Classification (W1 & Q1). Section 5 now includes CIFAR-10 and CIFAR-100, where our method outperforms DP-SGD at every budget from ϵ=0.25\epsilon=0.25 to 88, e.g., 98.4% versus 78.5% on CIFAR-10 (binary) at ϵ=1\epsilon=1.
  • Re-tuned DP-SGD (W2 & Q2, W4 & Q4). In both tasks, the learning rate, the number of updates, the clipping threshold, and the batch size are selected on public tasks (Appendices E and G). The candidate batch sizes are 32 and 128, the full private dataset, since large batches often help DP-SGD (De et al., 2022). Appendix E also explains why the selected learning rates of 10210^2 and 10310^3 are large.
  • Reference results (W3 & Q3). The pretrained model, public-only fine-tuning, and non-private fine-tuning attain FID 251.4, 230.4, and 106.6, compared with 137.4 for our method at ϵ=1\epsilon=1 (Appendix E).
  • Scope of the privacy guarantee (W5 & Q5). Section 5 now states that the guarantee applies to each 128-example task, not to the kk-NN construction or the joint release of all 100 models.

We would be grateful to know whether these revisions address your concerns, and we would be happy to clarify anything further before the discussion period ends. If they do, we would appreciate it if you could consider updating your score.

Reviewer_QSt6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes to train a hypernetwork on public dataset, and then apply it on perturbed embedding of private dataset. The intuition is that by restricting perturbations to dataset embedding space, the algorithm circumvents adding noise to other parts of the network, thus potentially offering better privacy-utility tradeoffs compared to gradient perturbations. Theoretical analysis for a synthetic linear regression task with "roughly low rank" true parameters confirm the intuition: under MLP-based hyper networks, while DP-SGD suffers from a parameter-estimation lower bound that scales quadratically with the input-dimension, the proposed method can reduce the error to approximately the intrinsic dimension of the true parameters. Experiments for LoRA fine-tuning of diffusion models on CIFAR-10 images confirm the benefits: in a fixed small-private-dataset setting, across different DP budgets, hyper networks (trained either on in-distribution CIFAR-10 public data or shifted-distribution Imagenet data) consistently generates outputs that have better better utility (FID score) compared to models fine-tuned via standard DP-SGD and other gradient-perturbation-based public-data-assisted private training methods.

Critical strengths
  1. Effectively adapting pretrained model on small private dataset is important, and gradient-based methods are known to suffer from curse of input dimensionality. The author propose a novel pipeline for applying hyper networks to private adaptation. The proposed privatized variant of hyper networks provably improves privacy-utility tradeoffs both theoretically for synthetic linear regression and practically for LoRA fine-tuning of diffusion models.
  2. The paper follows a rather complete structure: proposing method, providing intuition, theoretical analysis for the intuition, as well as experimental confirmation. The writing is clear in most places. The comparison baselines are also thorough, including most SoTA public-data-assisted gradient perturbation methods.
Critical weaknesses

Although the paper's title emphasizes that the observed improvement comes from avoiding iterative noise injection, I doubt how much of the observed improvement comes from avoiding iterative noise injection, as opposed to exploiting a low-dimensional representation learned from public data.

For the theoretical linear regression analysis, the gain mostly comes from exploiting a low-dimensional representation, and the prior Zhou et al.'s projected gradient descent algorithm on subspace estimated by public-dataset (still adding iterative noise similar to DP-SGD) can achieve the same asymptotic improvement.

For experiments, although they demonstrate clear improvements over the evaluated DP-SGD baselines and other prior baselines, it is unclear how much of the improvement is attributable to the hypernetwork architecture versus the significantly smaller dimension of the 128-dimensional privatized dataset representation. In particular, rank-1 LoRA still involves 40,704 trainable parameters, while recent parameter-efficient adaptation methods such as LoRA-XS and TinyLoRA demonstrate that substantially smaller trainable parameterizations are possible. A comparison with DP optimization in a similarly low-dimensional adaptation space, potentially leveraging public data to learn that space, would help clarify whether the proposed method offers benefits beyond dimensionality reduction.

Questions for authors

How much of the observed improvement comes from avoiding iterative noise injection, as opposed to exploiting a low-dimensional representation learned from public data? Would the method have benefits over DP optimization in a similarly low-dimensional adaptation space, potentially leveraging public data to learn that space?

4 comments
AuthorsOpenReview ↗

Response to Reviewer QSt6

We thank the reviewer for the positive assessment and for the precise question on the source of the improvement. We address it below; all revisions mentioned here are included in the revised PDF.

Source of the improvement

How much of the observed improvement comes from avoiding iterative noise injection, as opposed to exploiting a low-dimensional representation learned from public data?

In our method, adding noise only once and adding it to a low-dimensional representation are both made possible by training the hypernetwork on public tasks. Because the hypernetwork learns a map from datasets to parameters, the parameters can be generated from a single noisy, low-dimensional dataset embedding instead of being optimized with noisy gradients. We have revised the introduction to make this explicit.

The contribution of the low-dimensional representation alone can still be measured with Projected DP-SGD (Zhou et al., 2021), a baseline in our paper. It clips and perturbs gradients in a low-dimensional subspace built from public LoRA gradients, with its rank selected from {8,32,128}\lbrace8,32,128\rbrace on public tasks (rank 32 was selected). It thus exploits a representation learned from public data but still injects noise at every step. FID over the 100 private tasks (Appendix E):

ϵ=1\epsilon=1ϵ=4\epsilon=4
DP-SGD (re-tuned)369.0257.0
Projected DP-SGD245.5242.2
Ours137.4133.4

The public low-dimensional subspace therefore accounts for 53% of the gap between DP-SGD and our method at ϵ=1\epsilon=1 and 12% at ϵ=4\epsilon=4. The rest comes from the learned map and the single noise injection it enables.

Dimension of the privatized representation

... it is unclear how much of the improvement is attributable to the hypernetwork architecture versus the significantly smaller dimension of the 128-dimensional privatized dataset representation.

The 128-dimensional context token is indeed privatized, but the clipping and noise are applied earlier, to a 512-dimensional representation (Appendix E). This is larger than the 32 dimensions of Projected DP-SGD, so a smaller noise dimension does not explain our advantage.

Our ablation in Appendix F varies these two factors separately. Changing the noise dimension among 256, 512, and 1,024 changes FID by at most 1.27, whereas shrinking the hypernetwork with the dimension fixed at 512 increases FID from 137.4 to 147.7. Hence, our improvement does not come from a small noise dimension, and it depends on the capacity of the hypernetwork instead.

Smaller adaptation spaces

... recent parameter-efficient adaptation methods such as LoRA-XS and TinyLoRA demonstrate that substantially smaller trainable parameterizations are possible.

Would the method have benefits over DP optimization in a similarly low-dimensional adaptation space, potentially leveraging public data to learn that space?

Yes. Projected DP-SGD is DP optimization in a 32-dimensional adaptation space learned from public data, and our method attains a much lower FID at all four budgets where it was evaluated (ϵ\epsilon from 1 to 64; Appendix E). Smaller spaces such as LoRA-XS and TinyLoRA could reduce the dimension further. However, rank 8 did not outperform rank 32 in the public selection for Projected DP-SGD, so we do not expect a smaller space alone to close the gap.

Theoretical analysis

For the theoretical linear regression analysis, the gain mostly comes from exploiting a low-dimensional representation, and the prior Zhou et al.'s projected gradient descent algorithm ... can achieve the same asymptotic improvement.

We agree that, in the linear model, the gain comes from exploiting the low-dimensional structure of the task family. We also agree that, in this model, projected DP gradient descent on a subspace estimated from public data can exploit the structure. Our theorem does not claim that a single noise injection is necessary for this gain. It shows that a hypernetwork trained end to end on public tasks, without being given the structure, learns it and attains the improved rate with a single noise injection, while full-batch DP-SGD, which ignores the structure, cannot. Beyond the linear model, however, our method outperforms Projected DP-SGD by a wide margin, as shown above (FID 137.4 versus 245.5 at ϵ=1\epsilon=1). We have revised Section 4 to state that our analysis compares with full-batch DP-SGD, which does not use public data, and that methods using a public subspace (Zhou et al., 2021) can also exploit the structure in the linear model and are compared empirically in Section 5.


We would be happy to clarify any remaining concerns during the discussion period. We would greatly appreciate it if you could consider increasing the score once all concerns have been resolved.

AuthorsOpenReview ↗

Further revised PDF and summary of revisions

Thank you again for your positive assessment. We have uploaded a further revised PDF in which DP-SGD in Figure 2 is re-tuned at all budgets. Our method remains better at every budget from ϵ=1\epsilon=1 to 128128, where both are evaluated.

Our answer to your question and the related revisions are summarized below, with details in our response above.

  • Source of the improvement. Adding noise only once and adding it to a low-dimensional representation are both enabled by training the hypernetwork on public tasks, which we have made explicit in the introduction.
  • Comparison with low-dimensional DP optimization. Projected DP-SGD, which uses a subspace learned from public data but injects noise at every step, attains FID 245.5 at ϵ=1\epsilon=1, compared with 137.4 for our method (Appendix E), so a low-dimensional space learned from public data does not by itself explain our advantage.
  • Scope of the theoretical comparison. Section 4 now states that our analysis compares with full-batch DP-SGD, which does not use public data, and that methods using a public subspace, such as that of Zhou et al. (2021), can also exploit the structure in the linear model.

We would be glad to know whether this answers your question, and we would be happy to clarify anything further before the discussion period ends. If it does, we would appreciate it if you could take it into account in your final evaluation.

Thanks for the clarifications. I have remaining doubts on the comparison with low-dimensional DP optimization. Could the authors clarify exactly the number of trainable parameters of each approach? In my understanding, the gradient subspace dimension under projected DP training is not directly comparable to the 128-dimensional context token or the earlier 512-dimensional context tokens. I'm not fully convinced on how the authors draw the comparison "The public low-dimensional subspace therefore accounts for 53% of the gap between DP-SGD and our method at ε=1\varepsilon=1 and 12% at ε=4\varepsilon=4."

Thank you for the follow-up questions.

Number of parameters

All methods output the 40,704 LoRA parameters of the same model. They differ in what is optimized on private data, what receives noise, and what is learned from public data:

MethodOptimized on private dataNoised quantityParameters learned from public data
DP-SGD40,704 LoRA parameters40,704-dim gradient, every updatenone
Projected DP-SGD40,704 LoRA parameters, updated within a fixed 32-dim subspace (32 free coordinates)32-dim projected gradient, every update32-dim subspace basis from public gradients
Oursnone512-dim dataset embedding, oncehypernetwork (1,969,152 parameters)

In our method, the private data is not used for any optimization. It is only encoded into the noisy 512-dim embedding, which is input to the hypernetwork, and the 128-dim context token is computed from this embedding by a post-noise MLP. Neither dimension is a number of trainable parameters.

Dimension of the noised quantity

To clarify, our earlier comparison of 512 and 32 concerns only the number of coordinates that receive noise. In both methods, per-example vectors of this dimension are clipped, Gaussian noise is added to their aggregate, and the noisy vector is mapped into the LoRA parameter space. Projected DP-SGD adds noise to 32 coordinates at every update, whereas our method adds noise to 512 coordinates once. Since our method attains a much lower FID (137.4 versus 245.5 at ϵ=1\epsilon=1) while adding noise to more coordinates, a smaller noised dimension does not explain its advantage.

Within our method, Appendix F addresses your original question of the hypernetwork versus the dimension by varying the noised dimension and the hypernetwork size separately at ϵ=1\epsilon=1. Changing the dimension among 256, 512, and 1,024 changes FID by at most 1.27, whereas a smaller hypernetwork with the dimension fixed at 512 increases FID from 137.4 to 147.7. In these experiments, FID is more sensitive to the hypernetwork size than to the dimension.

Comparison with Projected DP-SGD

To be precise, the 53% at ϵ=1\epsilon=1 and 12% at ϵ=4\epsilon=4 are the fractions of the FID gap between DP-SGD and our method that Projected DP-SGD closes, computed as (DP-SGD − Projected DP-SGD)/(DP-SGD − ours). The remaining gap is large at both budgets. Projected DP-SGD performs DP optimization within a subspace learned from public data, with its rank selected from {8, 32, 128} on public tasks, and attains FID 245.5 at ϵ=1\epsilon=1 and 242.2 at ϵ=4\epsilon=4, compared with 137.4 and 133.4 for our method. Thus, reducing the dimension of DP optimization with a subspace learned from public data, even with the dimension tuned, does not close the gap to our method.

We hope this clarifies the comparison, and we would be happy to answer further questions.

Reviewer_n6F4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper supposedly improves on DP-SGD by using a hypernetwork (trained on a public dataset) to generate the weights of a target network. The key to the improvement is that the DP noise is only added once during the weight generation, as opposed to DP-SGD, which needs adds noise to the gradients at every optimizer update. In essence, the method takes a dataset as input, passes it through a neural network to generate embeddings, clips and perturbates these embeddings and finally passes them on for further processing to generate the weights---this being private due to post-processing property of DP. The method is shown to improve utility in a synthetic setting and also improving FID of a real-world LoRA fine-tuning image generation task.

Critical strengths
  1. The method shows substantial utility improvements in the field of image generation. The reason is easy to understand: adding noise once instead of iteratively can reduce the total amount of noise injected.
  2. The method enables private image generation at much lower ε\varepsilon than existing methods according to FID score (Figure 2).
  3. The paper has theory that shows that the method can perform better than DP-SGD. However, I was not able to properly check the theory. That being said, it seems legit to me.
Critical weaknesses
  1. The paper title and even the abstract makes the paper sound like a generic method, but its only real-world demonstration is image generation.
  2. In the discussion, other modalities than image generation are left as future work. However, including even an image classification task in the experiments would make the paper much more compelling. It is not clear, if this works outside of image generation.
  3. The image generation baselines use a model pretrained on ImageNet64, and the downstream task is CIFAR-10 which is very similar to ImageNet. The model should be evaluated on a proper out-of-distribution dataset.
  4. The baseline hyperparameter tuning could likely be improved, I have addressed these points individually in the questions for authors.
Questions for authors
  1. In Appendix Table 2, the learning rates for DP-SGD seem to be really large, in the order of 10210^2--10310^3. Furthermore, for example in the DP-SGD case, only two learning rates are tested. These makes me feel like the DP-SGD hyperparameter search is not adequate. Can you elaborate on these choices?
  2. How did you choose the batch size for DP-SGD? In general larger batch sizes work with DP-SGD [1], so your selection of B=32B=32 seems suboptimal.
  3. You seem to choose the clipping bound based on gradient norm quantiles, but this is private information. Can you elaborate why this does not violate privacy?
  4. How does the method do in comparison to non-DP iterative image generation methods?
  5. How does the method do in comparison to non-DP hypernetwork methods?

1 - "Subsampling is not Magic: Why Large Batch Sizes Work for Differentially Private Stochastic Optimisation" (Räisä et al., 2024) - https://arxiv.org/abs/2402.03990

8 comments
AuthorsOpenReview ↗

Response to Reviewer n6F4 (1/2)

We thank the reviewer for the careful reading and the constructive feedback. We address the concerns and questions below; all revisions mentioned here are included in the revised PDF unless stated otherwise.

W1 & W2: Evaluation beyond image generation

The paper title and even the abstract makes the paper sound like a generic method, but its only real-world demonstration is image generation.

... including even an image classification task in the experiments would make the paper much more compelling.

Following this suggestion, we have added image classification experiments to Section 5. The hypernetwork generates a linear classifier on frozen CLIP ViT-B/32 features, and each private task contains only 128 labeled examples, as in our generation experiments. The private tasks use class pairs of CIFAR-10 (binary) and classes of CIFAR-100 (10-way) that the hypernetwork never sees during training.

The baseline is DP-SGD that trains the same linear classifier on the same features and the same 128 examples. Its batch size (32 or 128), learning rate, number of updates, and clipping threshold are selected on public tasks only. With δ=10−5\delta=10^{-5}, accuracy (%) over 100 private tasks is:

TaskMethodϵ=0.25\epsilon=0.250.50.511224488non-DP
CIFAR-10Ours79.597.698.498.598.398.698.7
CIFAR-10DP-SGD58.666.078.590.896.097.598.1
CIFAR-100Ours16.523.134.352.861.867.075.1
CIFAR-100DP-SGD10.811.312.315.824.936.785.8

Our method outperforms DP-SGD at every budget, by up to 37.0 points on CIFAR-100. This shows that the benefit of our approach is not specific to image generation. The non-DP columns are our hypernetwork without noise and SGD without clipping or noise. We have also revised the abstract and the limitations to cover both tasks.

W3: Out-of-distribution private data

The image generation baselines use a model pretrained on ImageNet64, and the downstream task is CIFAR-10 which is very similar to ImageNet. ...

Adapting ImageNet-pretrained models to CIFAR-10 is common in DP image generation with public pretraining (Ghalebikesabi et al., 2023; Lyu et al., 2023; Lin et al., 2024). All methods start from the same pretrained model, so the closeness of the pretraining data applies equally to all of them, and the comparison remains fair.

Since our method relies heavily on the public tasks used to train the hypernetwork, we consider the shift between public and private data particularly important. In the new CIFAR-100 experiment, the private tasks consist of 20 classes that the hypernetwork never sees during public training, and our method reaches 52.8% versus 15.8% for DP-SGD at ϵ=2\epsilon=2. Thus, its advantage does not require the private tasks to follow the distribution of the public tasks.

We agree that a private domain far from the pretraining data, such as medical images, is an important further test, and we have added it to the limitations.

AuthorsOpenReview ↗

Response to Reviewer n6F4 (2/2)

W4 & Q1: Learning rates and tuning of DP-SGD

In Appendix Table 2, the learning rates for DP-SGD seem to be really large, ... only two learning rates are tested.

The values are correct. They are large because the gradients of rank-1 LoRA are small, with a median per-example norm of about 1.7×10−31.7\times10^{-3} on public tasks. Without noise, each DP-SGD update has norm at most the learning rate times the clipping threshold CC, so a learning rate of 10310^3 with C≈1.7×10−3C\approx1.7\times10^{-3} gives updates of norm at most 1.7. DP-AdamW, which normalizes the gradient scale, selects ordinary learning rates of 0.003–0.01.

We agree that two learning rates with a fixed CC were not enough. We therefore re-tuned DP-SGD on public tasks only. From public searches over the learning rate ({10,102,103}\lbrace10,10^2,10^3\rbrace, extended by one decade when an endpoint wins), the number of updates, and CC chosen from quantiles of public gradient norms, we fix a common set of candidates for all budgets. For each ϵ\epsilon, we compare these candidates with batch sizes 32 and 128 on public tasks and confirm the best ones on held-out public tasks, before a single evaluation on the 100 private tasks. FID (mean ±\pm SD over the 100 private tasks):

ϵ=1\epsilon=1ϵ=4\epsilon=4ϵ=1024\epsilon=1024
DP-SGD, submitted426.7 ±\pm 41.1338.8 ±\pm 48.1215.3 ±\pm 32.3
DP-SGD, re-tuned369.0 ±\pm 41.7257.0 ±\pm 20.7152.1 ±\pm 20.3
Ours137.4 ±\pm 24.3133.4 ±\pm 23.2—

Re-tuning improved DP-SGD at all three budgets, but at ϵ≤4\epsilon\le4 our method remains better by more than 120 in FID. We have updated Table 1 with these values. We are applying the same protocol to all budgets in Figure 2 and will upload a further revised PDF with the updated figure before the end of the discussion period.

Q2: Batch size of DP-SGD

How did you choose the batch size for DP-SGD? In general larger batch sizes work with DP-SGD [1], so your selection of B=32B = 32 seems suboptimal.

We had fixed B=32B=32 without tuning, and we agree that larger batches often help DP-SGD (Räisä et al., 2024). In the re-tuning above, the batch size is selected from 32 and 128, the full private dataset, on public tasks, and 128 was selected at all three budgets. Public selection also chose B=128B=128 in classification for CIFAR-100 at ϵ=1\epsilon=1 and 44.

Q3: Choice of the clipping threshold

You seem to choose the clipping bound based on gradient norm quantiles, but this is private information. ...

The choice of the clipping threshold uses no private data. The quantiles are computed on public tasks only (public task IDs 0–7), as stated in "Public hyperparameter selection" of Appendix E. All hyperparameters, including CC, are fixed on public tasks before the single evaluation on the 100 private tasks. The same holds for the clipping scale of our method, chosen from embedding norms on public data, and for both methods in the new classification experiments.

Q4: Non-DP iterative fine-tuning

How does the method do in comparison to non-DP iterative image generation methods?

We have added non-private fine-tuning with AdamW on the same 128 examples of each private task, with the same rank-1 LoRA and with its learning rate and number of updates selected on public tasks. It attains FID 106.6±21.7106.6\pm21.7, compared with 137.4137.4 for our method at ϵ=1\epsilon=1 and 251.1251.1 for DP-AdamW, which uses the same optimizer with clipping and noise. Thus our method closes most of the gap between DP-AdamW and non-private fine-tuning (about 79%).

Q5: Non-DP hypernetwork

How does the method do in comparison to non-DP hypernetwork methods?

The closest non-private hypernetwork is our model without noise, and the gap to it is the cost of privacy for our method. In classification, this cost is much smaller than that of DP-SGD. Without noise, our hypernetwork reaches 98.7% on CIFAR-10 and 75.1% on CIFAR-100, and per-task SGD reaches 98.1% and 85.8%. At ϵ=4\epsilon=4, our method loses 0.4 and 13.3 points from its non-DP version, whereas DP-SGD loses 2.1 and 60.9 points from non-DP SGD. Per-task SGD is better on CIFAR-100 without privacy, since the hypernetwork must transfer to unseen classes in a single forward pass, but under DP our method is far better.

For generation, we have not trained a non-private hypernetwork. However, the FID of our method stays between 132.8 and 137.4 for ϵ\epsilon from 1 to 128 (Figure 2), which indicates that the noise costs little at ϵ≥1\epsilon\ge1.


We would be happy to clarify any remaining concerns during the discussion period. We would greatly appreciate it if you could consider increasing the score once all concerns have been resolved.

AuthorsOpenReview ↗

Further revised PDF and summary of revisions

Thank you again for your review. We have uploaded a further revised PDF in which DP-SGD in Figure 2 is re-tuned at all budgets, as promised in our response to W4 & Q1. Our method remains better at every budget from ϵ=1\epsilon=1 to 128128, where both are evaluated, e.g., FID 132.8 versus 251.9 at ϵ=8\epsilon=8.

The main revisions in response to your review are as follows, with details in our response above.

  • Image classification (W1 & W2). Section 5 now includes CIFAR-10 and CIFAR-100, where our method outperforms DP-SGD at every budget from ϵ=0.25\epsilon=0.25 to 88, e.g., 52.8% versus 15.8% on CIFAR-100 at ϵ=2\epsilon=2.
  • Re-tuned DP-SGD (W4, Q1 & Q2). The learning rate, the number of updates, the clipping threshold, and the batch size are selected on public tasks at all budgets (Appendix E). Batch size 128, the full private dataset, is selected at every budget. Appendix E also explains why the selected learning rates of 10210^2 and 10310^3 are large.
  • Non-private reference (Q4). Non-private fine-tuning attains FID 106.6, compared with 137.4 for our method at ϵ=1\epsilon=1 (Appendix E).

We would be grateful to know whether these revisions address your concerns, and we would be happy to clarify anything further before the discussion period ends. If they do, we would appreciate it if you could consider updating your score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of revisions and additional experiments

We thank all reviewers for their careful reading and constructive feedback. Our paper proposes to generate model parameters from a private dataset with a hypernetwork trained on public tasks, which avoids the repeated noise injection of DP training by perturbing a dataset embedding only once. As the reviewers noted, this yields substantial utility improvements and enables private image generation at much lower ε than existing methods. Below we summarize our revisions. All revisions are included in the revised PDF and marked in blue. We are re-tuning DP-SGD at all budgets in Figure 2 with the same protocol, and will upload a further revised PDF with the updated figure before the end of the discussion period.

New experiments

  • Image classification. Our method generates a linear classifier on frozen CLIP features for CIFAR-10 (binary) and CIFAR-100 (10-way) tasks with 128 examples each, on classes unseen during public training. It outperforms DP-SGD tuned on public tasks at all budgets from ϵ=0.25\epsilon=0.25 to 88, e.g., 52.8% versus 15.8% on CIFAR-100 at ϵ=2\epsilon=2 (Section 5 and Appendix G; n6F4, Ftcg).
  • Re-tuned DP-SGD for generation. The learning rate, the number of updates, the clipping threshold, and the batch size (32 or 128, the full private dataset) are selected on public tasks only. The re-tuned FIDs at ϵ=1\epsilon=1 and 44 are 369.0 and 257.0, versus 137.4 and 133.4 for our method (Appendix E; n6F4, Ftcg). Table 1 already uses these values.
  • Reference results. On the same private tasks, the pretrained model has FID 251.4, public-only fine-tuning 230.4, and non-private fine-tuning 106.6, compared with 137.4 for our method at ϵ=1\epsilon=1 (Appendix E; n6F4, Ftcg).

Other revisions

  • Introduction: why the hypernetwork needs only a single noisy, low-dimensional dataset embedding (QSt6).
  • Section 4: the scope of the theoretical comparison, and its relation to Zhou et al. (2021) (QSt6).
  • Section 5: the scope of the privacy guarantee (Ftcg).
  • Appendix E: why DP-SGD uses large learning rates (n6F4, Ftcg).
  • Limitations: private data far from the pretraining distribution, such as medical images (n6F4).

We hope these revisions address the main concerns, and we would be happy to discuss any remaining questions.

AuthorsOpenReview ↗

Further revised PDF: re-tuned DP-SGD in Figure 2

We have uploaded a further revised PDF, in which DP-SGD in Figure 2 is re-tuned at all budgets with the protocol described in Appendix E. Our method remains better at every budget from ϵ=1\epsilon=1 to 128128, where both are evaluated, e.g., FID 132.8 versus 251.9 at ϵ=8\epsilon=8 and 133.1 versus 186.7 at ϵ=128\epsilon=128. Tables 4 and 5 in Appendix E give the DP-SGD values at all budgets. We are also re-tuning Public FT + DP-SGD with the same protocol. At ϵ=1\epsilon=1 and 44, its re-tuned FIDs are 368.6 and 231.6, compared with 137.4 and 133.4 for our method, and we will update its values in Figure 2 and Table 4 once the remaining budgets are complete.

AuthorsOpenReview ↗

Re-tuned Public FT + DP-SGD in Figure 2 and Table 4

We have uploaded a further revised PDF, in which Public FT + DP-SGD in Figure 2 and Table 4 is re-tuned at all six of its budgets, as promised in our previous comment. Its hyperparameters and batch size in the DP-SGD stage are selected on public tasks with the same candidate-comparison procedure as DP-SGD (Appendix E). Our method remains better than every baseline at every budget from ϵ=1\epsilon=1 to 128128 where both are evaluated. Against Public FT + DP-SGD, for example, the FIDs are 133.4 versus 231.6 at ϵ=4\epsilon=4 and 134.1 versus 213.9 at ϵ=64\epsilon=64.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.