acceptodds
Under review as a conference paper at ICLR 2027

The Wishart Projection Mechanism Is Almost Differentially Private: Guarantees and Limitations

Abstract

We introduce and analyse the *Wishart projection mechanism*, a randomised map of the form where , and study its differential privacy (DP) properties. For vector-valued queries , we prove non-asymptotic DP guarantees showing that Wishart randomness alone suffices, without requiring any additive noise. For matrix-valued queries, however, we establish a sharp separation: we prove that without additive noise the mechanism is not DP, and empirically supplement this theoretical result with a near-perfect membership inference attack (). As a concrete instantiation, we show that LoRA-style updates are an instance of the matrix-valued Wishart projection mechanism, implying that LoRA is not inherently private despite its initialisation randomness, and that empirical observations of reduced memorisation under LoRA should not be interpreted as DP guarantees. We then analyse a noisy variant and prove that the Wishart projection can *amplify* the resulting privacy guarantee: at a fixed noise multiplier, composing with a freshly resampled projection yields strictly stronger DP bounds than noise alone, both in the large- and small- regimes.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Differential Privacy, the Wishart Mechanism, LoRA

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_dteZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the differential privacy (DP) of the Wishart projection mechanism, which maps a vector or matrix-valued query f(S)f(S) to f(S)Mf(S)M, where MM is a randomly sampled Wishart matrix. The authors first show that, for vector queries, the mechanism satisfies approximate DP when neighboring query outputs are sufficiently "aligned". In contrast, for matrix queries, they prove that the mechanism is not DP, by exploiting algebraic constraints on the output induced by the symmetry of MM. The key observation is that, for a matrix input VV and output Y=VMY=VM, the matrix YV⊤YV^\top must be symmetric, whereas for a neighboring input V′V' that is not a scalar multiple of VV (assuming full rank), for Y′=V′M′Y'=V'M', Y′V⊤Y'V^\top is symmetric with probability zero. They relate this negative result to LoRA, observing that its first weight update can be expressed as a Wishart projection.

Next, the authors propose two noisy variants: adding Gaussian noise after or before projection. They show that the randomness in the projection can yield stronger DP guarantees than conventional Gaussian mechanism, in the large-rank and small-rank regimes, respectively.

Empirically, the paper shows near-perfect membership inference attacks against noise-free LoRA-style training, and compares the proposed noisy projection method with DP-SGD, DP-LoRA, and DP-LoRA-FA on classification tasks. Their method achieves accuracy generally comparable to the LoRA-based baselines.

Critical strengths

I find the negative result for matrix-valued Wishart projections technically clean. The proof identifies a simple structural obstruction to DP. This helps clarify why privacy guarantees for Gaussian sketches do not necessarily extend to Wishart-projected updates, as might be inferred from prior work such as Malekmohammadi and Farnadi.

(As a side note, the discussion of Malekmohammadi and Farnadi (2024) in Section 5 appears to refer to a later version of the same paper (February 2026), which explicitly claims formal DP guarantees. It may be helpful to update the citation to reflect this version.)

Critical weaknesses
  • I have two concerns about the correctness of the positive privacy results:

    • In Algorithm 2, the authors clip gradients with respect to the LoRA factor BB, which depend on the randomly sampled projection matrix AA. However, Theorem 5 assumes that the input to the projection mechanism is independent of the projection randomness. Thus, Theorem 5 does not directly justify the privacy accounting used for Algorithm 2 and the corresponding experiments.
    • In Lemma 16 (used to prove Theorem 4), the authors apply the classical Gaussian mechanism bound without requiring ε∥<1\varepsilon_\parallel<1, even though this bound is not generally valid for larger ε\varepsilon. This appears to invalidate the claimed privacy guarantee in Theorem 4 for some parameter regimes.

    (To be transparent, I used an LLM to identify potential technical issues and then went through them myself. I think both issues are valid and somewhat important to the correctness of the paper's results.)

  • I think the negative result is less significant than what the abstract and introduction seem to suggest. In particular, the paper makes strong claims such as "LoRA is not inherently private" and "the consequence for LoRA is immediate: without additional additive noise, neighbouring datasets can yield non-overlapping support, ruling out any (finite-ε\varepsilon, finite-δ\delta) DP guarantee."

    However, LoRA usually refers to a fine-tuning procedure involving multiple optimization steps. The paper only establishes that the first update of standard LoRA, where B0=0B_0=0 and the gradient is independent of the randomly initialized A0A_0, can be expressed as a Wishart projection and is therefore not DP in the worst case. In subsequent steps, both AtA_t and BtB_t evolve, and the gradients may depend on the adapter parameters from previous steps, so the same argument no longer directly applies. More importantly, in a standard use case, only the final model WTW_T is released, rather than the intermediate weights W1W_1. As the authors acknowledge in Section 3.2, their impossibility result does not directly apply to this setting. This limits the significance of the result for practical LoRA training, which is my main reservation about the paper's contribution to an ICLR audience.

  • A minor point: Regarding M2, the authors show that the randomness in the projection can yield stronger DP guarantees than the standard Gaussian mechanism. However, I'm not sure how meaningful this comparison is in terms of utility, since M2 introduces additional approximation error through projection, and thus may have substantially different use case from the standard Gaussian mechanism.

Questions for authors

See the Weakness section.

5 comments

Thank you for your careful reading. Let us first address your concerns about the correctness of the privacy guarantees in Theorems 4 and 5.

Algorithm 2 and Theorem 5. The reference to Algorithm 2 in the experimental section was an oversight. This issue was also pointed out by another reviewer and we apologise for this mistake. The experiments evaluating Theorem 5 implement M2M_2: they compute and clip the per-example full-weight gradients, add Gaussian noise, and then apply a fresh projection AtA_t. The clipped input is therefore independent of AtA_t, as Theorem 5 requires.

We only included Algorithm 2 to illustrate how the projected update can be computed directly in LoRA coordinates without materializing full-weight gradients when clipping is inactive. With active clipping in those coordinates, the multiplier depends on AtA_t even though the threshold is fixed. We will clarify this distinction and correct the experimental cross-reference.

Lemma 16 and Theorem 4. We presented the classical Gaussian calibration because it gives a simple closed-form bound for 0<ε∥<10 < \varepsilon_\parallel < 1. The same argument extends to any ε∥>0\varepsilon_\parallel > 0 using the analytic Gaussian calibration of Balle and Wang 2018, which requires solving a scalar equation numerically and is therefore less compact to present. Our experiments already use the analytic calibration. We will add a remark clarifying both the extension and the calibration used in our experiments.

Scope of the negative result. Proposition 2 shows that the first noise-free update can violate DP, ruling out privacy for any released transcript that includes it. We will make that scope explicit in the abstract and introduction. However, we want to highlight that standard privacy accounting for DP-LoRA and DP-SGD composes guarantees for individual updates and the first-step result therefore shows why standard composition cannot certify its privacy.

To our knowledge, there is no general privacy characterization for releasing only the final model from standard LoRA. Establishing such a characterization across loss functions would be a substantial research contribution in its own right and is beyond the scope of this paper. However, a simple worst-case example showing that the support separation in Proposition 2 can persist through every training step is the following: Consider neighboring one-record datasets with losses ℓ0(W)=0\ell_0(W)=0 and ℓ1(W)=W11\ell_1(W)=W_{11}. The zero loss leaves WT=W0W_T=W_0. For ℓ1\ell_1, let at=Ate1a_t=A_te_1 and bt=Bt⊤e1b_t=B_t^\top e_1. Gradient descent gives at+1=at−ηbt,bt+1=bt−ηat.a_{t+1}=a_t-\eta b_t,\qquad b_{t+1}=b_t-\eta a_t. Since b0=0b_0=0 and a0≠0a_0\neq 0 almost surely, ata_t and btb_t are nonzero and point in opposite directions for all t≥1t\geq 1. Thus (WT−W0)11=bT⊤aT<0(W_T-W_0)_{11}=b_T^\top a_T<0 almost surely under ℓ1\ell_1. This event has probability 00 under ℓ0\ell_0 and 11 under ℓ1\ell_1, so releasing only WTW_T violates DP for any δ<1\delta<1.

For a more realistic setting we assessed final-model leakage empirically on CIFAR-10 (see Table 1 in the paper). A membership inference attack using only the final model achieves ROC-AUC above 0.99, showing that membership is effectively exposed for the tested examples even when all intermediate updates are withheld.

Related work (Malekmohammadi and Farnadi): We cited Malekmohammadi and Farnadi's 2024 preprint because it motivated our study of LoRA without additive noise. We omitted the February 2026 revision because we believe its privacy proof has a gap. It bounds privacy for a sketch of the form Z⊤VZ^\top V, then reuses the same random ZZ to reconstruct the update. The privacy bound for the sketch alone does not justify that step. Our impossibility result gives a counterexample for the noise-free update ZZ⊤VZZ^\top V. We will cite both versions and explain the distinction.

We hope these clarifications address your concerns about the privacy guarantees and show why the negative result matters in practice, even when only the final model is released.

Thank you again for your careful review. As the discussion period ends in two days (November 18 at 11:59pm AoE), we would like to gently remind you of our response above.

We hope our response has clarified the intention behind Algorithm 2 and Lemma 16, as well as the scope of our negative result. We will revise these parts in the paper to make them clearer. If any questions remain, please let us know and we will be happy to discuss further.

Thank you for the detailed clarifications regarding Algorithm 2/Theorem 5 and Lemma 16/Theorem 4. These address my main soundness concerns. Thank you also for explaining the citation to the earlier version of Malekmohammadi and Farnadi.

I also appreciate the additional multi-step counterexample. I agree that it establishes that releasing only the final LoRA model can fail DP in the worst case, and therefore addresses part of my original concern about the scope of the negative result.

My remaining reservation concerns the practical scope of the contribution. Given the paper’s motivation through LoRA, I remain less convinced that the Wishart analysis provides sufficient insight into model privacy in standard multi-step LoRA (beyond the new example ruling out an unconditional privacy guarantee). I understand that a general characterization is beyond the scope of this paper; my concern is whether the established result is sufficiently significant for ICLR. I therefore lean toward retaining my score.

Reviewer_aq3WOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a projection mechanism derived from the Wishart distribution, and presents its differential privacy (DP) properties for vector and matrix-valued queries. The motivation arises from related prior work on another class of projections: the Johnson-Lindenstrauss transform, which gives provable approximate DP guarantee without adding noise. The paper presents positive results for the vector-valued case with non-asymptotic DP guarantees built on certain minimum alignment requirements, and negative results for the matrix-valued case. The theoretical claims for matrix-valued queries are supported by membership inference attacks (MIA) on real-world datasets. In addition, the paper proposes two mechanisms to privatize the matrix-valued Wishart projections for large and small ranks. The authors prove that the proposed DP mechanisms can amplify privacy compared to simple additive Gaussian noise mechanisms. Lastly, through their small-rank results, the authors explain that low-rank adaptation (LoRA) does not provide inherent DP guarantees.

Critical strengths

The main idea to investigate Wishart projection in the context of DP is interesting and the findings can be considered a noteworthy contribution. For vector-valued functions, the assumption of minimum alignment on the data is straightforward, and the technique to improve alignment presented in Appendix C.1 seems reasonable. For matrix-valued functions, the relation between the proposed projection mechanism and LoRA, and the concrete theoretical statements on their lack of inherent DP properties are quite interesting. The paper is decently organized and the technical proofs seem correct, though difficult to follow at times due to notational overload, as well as lack of clarification and coordination of some theorems and lemmas.

Critical weaknesses
  • My primary concern lies in the feasibility of the privacy claims for vector-valued functions. The paper states that “Wishart randomness alone suffices” to prove DP. However, the minimum alignment ρ\rho, which is entirely governed by the dataset at hand, has a strict lower bound requirement in Theorem 1. If the data is not well aligned, Wishart projection alone cannot satisfy DP. In Figure 1, even for ρ=0.95\rho = 0.95, the lowest value of ϵ\epsilon is in the hundreds. The technique to increase ρ\rho in C.1 involves adding noise. Moreover, Lemma 12 has no proof, which further weakens the paper’s claim that Wishart vector projection is private with no noise. Unlike the matrix-valued case, there is no comparison of the privacy of Wishart vector projection against Gaussian mechanism. Addressing these issues and phrasing the claims accordingly would strengthen the paper.

  • The paper claims that the noisy variant of the Wishart matrix projection mechanism successfully holds off MIA. However, the results in Table 3 have an effective ϵ≈136.05\epsilon \approx 136.05, which is prohibitively high and essentially non-private.

Questions for authors

Please refer to the concerns mentioned above.

Additional minor comments:

  • Lemma 6 has no proof or citation
  • Additional proof (“Use the Kronecker-vec identity”) with no statement after the proof of Lemma 7
  • Reference missing (“??”) in Appendix I.3
5 comments

Thank you for your careful reading. We will clarify when Wishart projection alone can provide privacy and what the membership inference experiments establish.

Minimum alignment in Theorem 1. The lower bound on ρ\rho reflects a geometric limitation inherent to the Wishart projection. For example, if two neighboring unit vectors point in opposite directions, their Wishart-projected outputs lie in opposite half-spaces almost surely, so they are perfectly distinguishable. This obstruction persists regardless of how tightly we analyze the privacy loss. However, we want to note that other projection-based privacy guarantees also rely on geometric conditions. For instance, the well-known JL privacy result of Blocki et al. requires sufficiently large input singular values, which their applications enforce through preprocessing. Similarly, Appendix C.1 describes an optional preprocessing step that can improve alignment by adding noise before projection when the original query directions are poorly aligned.

Figure 1 We agree that the certified ε\varepsilon is large at ρ=0.95\rho=0.95. What makes the guarantee interesting is that Wishart projections already arise in practical ML workflows independent of privacy. For example, in LoRA-style updates, low-rank factorization is used to save computation and memory. Theorem 1 shows that, for sufficiently aligned vector-valued queries, this same kind of projection can provide some privacy “for free,” in the sense that the guarantee comes from randomness already present in the algorithm for non-privacy purposes.

Further we want to higlight that the minibatch averages used in SGD can have much stronger alignment than ρ=0.95\rho=0.95. Section 3.1 shows that if each per-example gradient has norm at most LL and every minibatch average has norm at least c0>0c_0>0, then 1−ρ≤8L2/(c02n2)1-\rho\leq 8L^2/(c_0^2n^2) for batch size nn. For example, with n=1024n=1024 and c0=L/10c_0=L/10, the bound gives ρ≥0.9992\rho\geq 0.9992 uniformly over neighboring minibatches.

Comparison with the Gaussian mechanism. In the matrix case, both mechanisms use additive Gaussian noise, so the Gaussian mechanism is a natural baseline for measuring how much the projection reduces the additive noise needed for a given privacy budget. The vector case uses no additve noise, so this type of comparison does not apply. More importantly, the vector result asks a different question. When a Wishart projection is already used in training to reduce computation and memory, can its existing randomness also provide privacy? Theorem 1 shows that it can under the alignment condition. So even if we matched the privacy budget of the Gaussian mechanism to the vector result, it is unclear how to compare utility while accounting for the lower computation and memory costs of the projected mechanism.

Interpretation of Table 3 We included Table 3 to illustrate the empirical implications of our theoretical result on disjoint supports. Without additive noise, neighboring inputs can yield output distributions with disjoint supports. Adding Gaussian noise makes those distributions overlap and substantially weakens the tested membership inference attack. ROC-AUC is above 0.990.99 for noise-free LoRA in Table 1 and ranges from 0.620.62 to 0.870.87 with noise in Table 3. We agree that "the attack no longer succeeds" was misleading. We meant that the output distributions are no longer disjoint and that the tested attack becomes less effective even for very little additive noise. We will revise the text accordingly and clarify that the reported ε≈136.05\varepsilon \approx 136.05 provides a weak worst-case guarantee.

Regarding the additional comments:

  1. The proof of Lemma 12 was inadvertently omitted. We will restore it.
  2. Lemma 6: We will cite Vershynin’s Corollary 5.35
  3. We will add the Kronecker–vec identity to the mathematical preliminaries and cite Henderson and Searle (1981, Eq. 4).
  4. we will replace the placeholder in Appendix I.3 with a reference to Figure 2

Thank you again for your careful review. As the discussion period ends in two days (November 18 at 11:59pm AoE), we would like to gently remind you of our response above.

We hope our response has addressed your concerns about the privacy guarantee and the minimum alignment, and that our clarifications on Table 3 have cleared up the confusion about its implications. If any questions remain, please let us know and we will be happy to discuss further.

I'd like to thank the authors for their detailed response. I appreciate the clarification of Table 3, and also for addressing the minor comments. Although, it'd be helpful to see a proof sketch of Lemma 12, since I cannot evaluate it from the authors' claim to restore it in the final version.

While my primary concerns were partially addressed, I am not convinced that the minibatch argument and the analogy to Blocki et al. are sufficient. Specifically, the bound ρ≥0.9992\rho \ge 0.9992 relies on every neighboring minibatch average having norm at least L/10L/10, which is unlikely to hold near convergence. Evaluating Theorem 1 at ρ=0.9992\rho = 0.9992 gives me ϵ≈2.9\epsilon \approx 2.9 for single release, before comprising over training steps. As for the comparison with Blocki et al., it does not quite carry over because the singular-value condition is enforced at preprocessing without involving additive noise. This is exactly why I think the claim “Wishart randomness alone suffices” needs to be stated with care in both the abstract and introduction.

If the authors provide the proof sketch and state the requirements for their claims explicitly, I'd be willing to reconsider my final score.

Reviewer_sB8rOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the intrinsic DP properties of Wishart random projections, with LoRA training as a concrete instantiation. The paper shows that Wishart randomness alone can provide approximate DP for vector-valued queries, but fails to provide DP for general matrix-valued queries. The paper then studies two variants and derive privacy guarantees showing that random projection can improve privacy bounds compared with Gaussian mechanisms alone in certain settings. The theoretical results are supported by experiments.

Critical strengths

The paper has the following strength.

(i) It provides both positive and negative results for intrinsic DP properties of Wishart mechanisms. Proposition 2 shows that the output distributions for neighboring inputs can have disjoint supports, which explains why Wishart randomness alone does not necessarily provide DP for matrix-valued cases. This is a useful theoretical results that can help us understand the intrinsic privacy limitations of, e.g., LoRA training.

(ii) The paper provides non-trivial mathematical analysis of Wishart mechanism based on Wishart random projection. The vector-valued intrinsic DP privacy guarantee and the analyses of matrix-valued mechanisms with additive Gaussian noise with privacy amplification are technically interesting.

Critical weaknesses

W1:

My main concern is whether the privacy guarantees in Theorem 1 can give us reasonable and acceptable utility at meaningful DP privacy budgets.

According to Theorem 1, the rank rr is the adjustable privacy parameter for the noise-free Wishart mechanism. However, rank also directly affects approximation quality and computational efficiency. In addition, the certified privacy bound is non-monotonic in the rank rr, and increasing the variance of the underlying Gaussian factors does not improve privacy (because it only rescales the output). It is very likely that there would be potentially substantial distortion at small ranks, particularly in high-dimensional settings.

Therefore, one main potential issue is that ranks satisfying a desired DP privacy budget may not provide acceptable utility, while ranks selected for acceptable utility may not satisfy the desired privacy requirement.

The paper does not establish whether the noise-free Wishart mechanism offers a better or similar achievable privacy-utility trade-off than the standard DP applied at the output.

W2:

In Theorem 5, the mechanism input difference is independently fixed of the random projection. However, Algorithm 2 clips per-example graidents after projection into the LoRA factor space. As a result, the clipping coefficients depend on the sampled matrix AA, and the resulting update is not generally equivalent to projecting the original clipped graidents.

I think that it would be better for the authors to clarify whether Theorem 5's beta-tail sensitivity bound applies to the implemented Algorithm 2. The authors should establish the appropriate privacy guarantee and justify the accountant used in their experiemnts.

W3:

Regarding the claimed privacy amplification, first, the amplification is not necessarily specific to Wishart matrices. Gaussian perturbation already provides DP, while the subsequent random projection can strength the guarantee by reducing the effective sensitivity. This is a legitimate amplification effect. But, it is an additional information-reducing operation. The authors should make the claim more accurate and clear by avoiding an impression that it is an independent alternative to standard Gaussian privatization. In addition, Theorem 5 also applies to orthonormalized random projections. Thus, the amplification primarily comes from random sensitivity reduction rather than some specific properties of Wishart, such as its eigenvalue structure.

Moreover, the Gaussian mechanism baseline in the comparison is actually conservative. In particular, the comparison is against a two-sided Gaussian privacy-loss tail rather the exact Gaussian mechnaism of standard DP privatization. Therefore, showing a smaller privacy bound than this baseline may not establish an improvement over tighter Gaussian mechanism. The authors should clarify whether the claimed quantitative advantage remains significant under tighter Gaussian mechanism.

Theorem 5 establishes the same conditional privacy-loss characterization for the Wishart mechanism and its orthonormalized-projection counterpart. It would be useful to clarify whether the Wishart structure provides any additional advantage in the privacy–utility trade-off compared with ordinary random projection under appropriately matched scaling and privacy budgets.

W4:

Theorem 1 establishes approximate-DP guarantees for vector-valued Wishart projections under unit-norm and strong alignment assumptions. However, using randomness inherent in random linear transformations can provide DP without additional additive noise has already been established in the literature, such as the work on Johnson–Lindenstrauss transforms (e.g., Blocki et al., 2012). While Theorem 1 provides a new analysis for a different random transformation, its significance beyond extending an existing privacy principle remains unclear.

What useful privacy capability or advantage does the Wishart mechanism offer over existing random-projection-based approaches? This question is especially relevant given the unresolved privacy–utility concerns pointed out in W1.

W5:

Theorems 3 and 4 rely on restrictive geometric assumptions that may not hold for practical neural network gradients. The experiments are constructed to satisfy certain conditions, and it remains unclear how these assumptions can be satisfied or verified in general applications.

Questions for authors

Please see comments in Critical Weaknesses.

3 comments

Thank you for your careful reading. Below, we provide brief explanations addressing your questions, and we hope these clarify your concerns.

W1 Privacy-utility tradeoff of Theorem 1, and hard to get a target privacy level by changing the rank rr.

Indeed, rank couples privacy and utility, and our guarantees may not yield a favorable trade-off when neighboring query outputs are poorly aligned. Our main contribution in theorem 1 is to be the first to answer the question whether any privacy can be expected at all when they are well-aligned. Specifically, our contribution is to characterize the privacy provided by Wishart randomness, which arises naturally in algorithms motivated by memory and computational efficiency, such as LoRA. Under favorable alignment conditions, this existing randomness can provide formal privacy guarantees without additional noise, giving almost "free privacy".

W2 updates in Algorithm 2 depends on AA and whether Theorem 5's bound applies to the implemented experiments

The reference to Algorithm 2 in the experimental section was an oversight. The experiments evaluating Theorem 5 implement M2M2: they compute and clip the per-example full-weight gradients, add Gaussian noise, and then multiply the noisy gradient by Mt=At⊤AtM_t=A_t^\top A_t, with a freshly sampled AtA_t. Conditional on the preceding training history, the clipped input is therefore independent of AtA_t, as Theorem 5 requires.

Algorithm 2 was included to illustrate how the projected update can be computed directly in LoRA coordinates without materializing full-weight gradients when clipping is inactive. With active clipping of ∇BL(B)At⊤\nabla_B L(B) A_t^\top, the resulting matrix Clip(∇BL(B)At⊤)AtClip(\nabla_B L(B) A_t^\top)A_t is no longer a data-dependent matrix multiplied by a random Wishart matrix, and hence Theorem 5 may not apply directly. We will clarify this distinction and correct the experimental cross-reference.

W3

  • Role of Gaussian noise and projection: Gaussian perturbation provides the initial differential privacy guarantee. The subsequent random projection strengthens the guarantee by reducing the effective sensitivity of the released output. We will clarify this interpretation of privacy amplification in the revision.
  • Scope of Theorem 5: We agree that Theorem 5 applies beyond Wishart matrices. The bound depends on the sensitivity reduction from projecting onto an independently sampled, Haar-random rr-dimensional subspace of Rd\mathbb{R}^d. We will add a general statement for Haar-random orthogonal projectors to the appendix.
  • Comparison with exact Gaussian calibration: We used the same two-sided privacy-loss bound for both the Gaussian baseline and Theorem 5 for clarity and notational simplicity of the comparison. The Gaussian baselines in Figures 2 and 3 are computed by numerically inverting the exact Gaussian privacy profile (i.e., analytical Gaussian calibration). As shown in the figures, our bounds yield smaller ε\varepsilon at the same sensitivity, noise level, and δ\delta. Thus, the reported privacy advantage already holds against the analytically calibrated Gaussian baseline. We will clarify this distinction in the revised text and figure captions.
  • Privacy–utility trade-off at matched scaling: Theorem 5 gives the same conditional privacy-loss characterization for Wishart projection and its orthonormalized counterpart. The Wishart eigenvalues rescale both the projected signal and the projected noise, so they can affect downstream utility. Whether this rescaling improves utility at matched rank, scaling, and privacy budgets remains open. Our large-rr analysis addresses a separate question. When r≥dr\geq d, the Wishart matrix is full rank almost surely. Under the stated geometric conditions, residual Wishart randomness can strengthen the privacy guarantee even after accounting for what the unchanged columns reveal, without reducing the output dimension.

W4 The unit norm assumption and the alignment assumptions are too strong in Theorem 1. Blocki et al 2012 already established JL transformation provides privacy by itself

Blocki et al. established that Gaussian JL sketches can provide differential privacy under suitable spectral conditions. When r<dr<d, the sketch Rf(S)Rf(S) maps the query from Rd\mathbb R^d into Rr\mathbb R^r. Our Wishart mechanism instead applies the rank-rr map r−1R⊤Rr^{-1}R^\top R from Rd\mathbb R^d to itself, so the output lies in a low-dimensional subspace while retaining the original query coordinates, as in LoRA-style updates.

We would also like to highlight, privacy of the JL transformation (with Gaussian sketching) Rf(S)Rf(S) from Blocki et. al. does not imply privacy of R⊤Rf(S)R^\top Rf(S) of Wishart mechanism. This is because pre-multiplying their Rf(S)Rf(S) with the same hidden random matrix RR again gives away all DP guarantees.

Further advantages over existing random-projection-based approaches: As detailed in the previous response, we analyze not only the setting where r<dr < d but also the large-rr case, when the dimension-reduction effect of random projection vanishes and the privacy amplification is purely due to the inherent randomness. To the best of our knowledge, no previous work provides such an analysis.

W5 Assumptions in Theorem 3 and 4 may not hold for practical NN gradients

As the reviewer mentions, these assumptions are not guaranteed for general neural-network gradients. We show in the discussion below theorem 1, that adding a small amount of noise can satisfy the conditions of thorem 1. Theorem 5 additionally, does not require these assumptions at all and can applied to NN gradients. Theorems 3 and 4 requires conditions similar to theorem 1. For specified neighboring gradient matrices, the geometric quantities can be evaluated directly. Such geometric assumptions also appear in prior work: for example, the privacy analysis of the Johnson–Lindenstrauss transform by [Blocki et al. (2012)] relies on a lower bound on the minimum singular values of neighboring input matrices. Propose-Test-Release offers a potential framework for privately certifying sufficient conditions before release [Dwork and Lei, 2009], but applying it here requires a suitable stability test and a proof covering all geometric conditions. Developing an efficient certification procedure for general neural-network gradients remains beyond the scope of our current results.

Dwork and Lei. Differential privacy and robust statistics. 2009.

Thanks to the authors for the detailed responses.

The rebuttal has addressed several of my concerns, particularly the applicability of Theorem 5 to the experimental implementation. The clarification regarding the actual implementation and the planned correction to Algorithm 2 are helpful. I also appreciate the explanations of the Gaussian baseline comparisons and the differences from existing random-projection approaches.

Some concerns remain, particularly regarding the privacy–utility trade-off, practical applicability, and the significance of certain theoretical improvements. I would encourage the authors to clearly acknowledge the following limitations in the final version:

  1. Theorem 1: The privacy guarantee does not necessarily lead to a favorable privacy–utility trade-off, and achieving a target privacy level by adjusting the rank can be difficult.
  2. Theorems 3–4: The required geometric assumptions may not hold for general neural-network gradients, and an efficient procedure for verifying these conditions in practical applications has not been established.
  3. Theorem 5: The small-rr privacy amplification is not specific to Wishart matrices but applies more generally to independent Haar-random projections. Whether Wishart projection provides additional practical advantages at matched privacy budgets remains unclear.
  4. Experimental implementation: The actual implementation should be clearly distinguished from Algorithm 2, including the implications of full-weight gradient clipping for computational efficiency.

I believe these clarifications would help readers better understand the scope, contributions, and limitations of the results.

In my opinion, this paper provides several interesting and nontrivial theoretical results and may have potential values for future research. I believe that this paper meets the standard of ICLR, and I would like to increase my score from 2 to 3.

Reviewer_EahzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the Wishart projection mechanism, which multiplies a query f(S)f(S) by a random matrix Z⊤ZZ^\top Z, where ZZ is a Gaussian matrix. When f(S)f(S) is vector-valued, it derives DP guarantees under alignment assumptions. For matrix-valued queries, the paper derives an impossibility result proving that Wishart randomness alone is not sufficient for privacy. More generally, the paper studies privacy amplification through random projections. It considers the mechanisms (M1) MV+ΞMV + \Xi and (M2) M(V+Ξ)M(V + \Xi) where Ξ\Xi is an independent Gaussian matrix, and proves amplification in the small and large rr regimes.

Critical strengths
  • Interesting problem and insights. The paper contributes to the privacy amplification literature. It asks whether randomness already present in an algorithm can improve privacy beyond the guarantees offered by standard mechanisms. This line of work is valuable because it allows for a tighter characterization of the privacy/utility trade-off of such mechanisms. The distinction between vector-valued and matrix-valued queries is useful.
Critical weaknesses
  • The positive DP result for vector-valued queries has a restricted scope. Theorem 1 assumes that ∥f(S)∥=1\|f(S)\| = 1, arguing that such a property can be achieved by normalization. However, normalization is data dependent and changes the released statistic. Restoring the original magnitude would require a separate privacy analysis. Therefore, the result cannot be interpreted as a general DP guarantee for vector-valued queries, but only as a DP guarantee in a restrictive class of queries.
  • Theorems 3 and 4 depend on restrictive geometric conditions. Theorem 3 and 4 leverage an adjacency relation requiring alignment and structural condition beyond bounded sensitivity. It is not obvious how they relate to classical DP adjacency notions, and do not seem to be guaranteed by simple clipping. The paper does not demonstrate that their geometric restrictions hold in practical settings or under the usual Hamming adjacency notion of DP.
Questions for authors
  • Lemma 1 requires ε∈(0,1)\varepsilon \in (0,1), but Equation (4) and Theorem 4 do not state this condition for ε∥\varepsilon_\parallel. Is it a missing assumption or I am missing a more general result?
  • Can the authors explain how the adjacency notion of this paper relates to the classical Hamming adjacency notion of DP? For classical mechanism, are the sensitivities comparable?
2 comments

Thank you for your careful reading. Below, we provide brief explanations addressing your questions, and we hope these clarify your concerns.

The positive DP result for vector-valued queries has a restricted scope

We agree that normalization changes the released statistic and that Theorem 1, as currently stated, applies to unit-norm queries. However, this assumption is made for simplicity and it is relatively simple to relax to allow non-unit-norm vectors. In particular, suppose that f(S)≠0f(S)\neq 0 and 1κ≤∥f(S)∥2∥f(S′)∥2≤κ\frac{1}{\kappa}\leq\frac{\Vert f(S)\Vert_2 }{\Vert f(S')\Vert_2 }\leq\kappa

for all neighboring datasets S,S′S,S', where κ≥1\kappa\geq1. Then define ρ\rho to be the angle between the normalized vectors. Let εunit norm\varepsilon_{\mathrm{unit\ norm}} denote the privacy bound given in Theorem 1 for the unit-norm case. Under the remaining assumptions of Theorem 1, the Wishart projection mechanism is (εκ,3η)(\varepsilon_\kappa,3\eta)-DP for any

\varepsilon_{\mathrm{unit\ norm}} +\frac{\nu+\Delta}{2(\rho-K)}(\kappa-1) -\frac{\nu}{2}\log\kappa.$$ Thus, relaxing the unit-norm assumption introduces an additive penalty in the privacy bound. This penalty depends on how far $\kappa$ deviates from 1, and the original unit-norm bound is recovered at $\kappa=1$. The proof follows the same density-ratio argument as Theorem 1, accounting additionally for the ratio of the two input norms. We will include this extension and its proof in the appendix. From an application perspective, normalization is also natural when only the direction of the query matters. For example, in private retrieval (Application C.2), normalizing the queries preserves the ranking of all retrieved items, since normalization scales their similarity scores by the same positive constant. Hence, the unit-norm assumption is appropriate for such applications. We will revise the discussion preceding Theorem 1 to clarify that normalization is a choice of query, rather than a transformation that preserves arbitrary vector-valued statistics. **Theorem 3 and 4 depend on restrictive geometric conditions** We thank the reviewer for their question. First, we would like to clarify that Theorems 3 and 4 use the standard Hamming adjacency notion for the underlying datasets, so neighboring datasets $S, S'$ differ in one record. The additional notion of one-column adjacency is introduced only as an analytical tool for proving the privacy guarantee, meaning Theorem 4 bounds privacy loss for a single-column change (between $f(S)$ and $f(S')$) and Theorem 3 composes these bounds to obtain the gurantee for Hamming-neighboring datasets. We also provide a concrete setting satisfying all the conditions in Appendix I.3. However, as we also mention in the paper, the privacy guarantees are not unconditional and hold under additional geometric assumptions on the queries $f$ (which are similar to the assumptions in the vector case). Such geometric assumptions also appear in other random projection works: for example, the privacy analysis of the Johnson–Lindenstrauss transform by [Blocki et al. (2012)] relies on a lower bound on the minimum singular values of neighboring input matrices. However, the neighbouring notion is the standard hamming distance. We will clarify this distinction in the revised manuscript. **Lemma 1 requires $\epsilon\in (0, 1)$** The reviewer is correct that we presented the classical Gaussian calibration. We chose to present this version of the Gaussian Mechanism because it gives a simple closed-form bound for $0 < \varepsilon < 1$ (this is also common in various papers in DP). However, the argument readily extends to any $\varepsilon > 0$ using the analytic Gaussian calibration of Balle and Wang 2018, which requires solving a scalar equation numerically and is therefore less compact to present. Our experiments already use the analytic calibration. We will add a remark clarifying both the extension and the calibration used in our experiments. **Comparison between the alignment assumption and the sensitivity requirement in the classical mechanisms** Our underlying notion of neighboring datasets is the standard Hamming adjacency: two datasets are adjacent if they differ in one record. The additional conditions in Definition 4 are about the geometry of the corresponding query functions $f$. For unit-norm queries, our conditios and the classical $\ell_2$-sensitivity bound are interchangeable descriptions of neighboring-output geometry. Specifically, letting $\rho$ denote the minimum alignment over neighboring datasets and $\Delta_2$ denote $\ell_2$ sensitivity, then $\Delta_2^2=2\bigl(1-\rho\bigr).$

Thanks for the answer which addresses my concerns. I decide to maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.