- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies the differential privacy (DP) of the Wishart projection mechanism, which maps a vector or matrix-valued query to , where is a randomly sampled Wishart matrix. The authors first show that, for vector queries, the mechanism satisfies approximate DP when neighboring query outputs are sufficiently "aligned". In contrast, for matrix queries, they prove that the mechanism is not DP, by exploiting algebraic constraints on the output induced by the symmetry of . The key observation is that, for a matrix input and output , the matrix must be symmetric, whereas for a neighboring input that is not a scalar multiple of (assuming full rank), for , is symmetric with probability zero. They relate this negative result to LoRA, observing that its first weight update can be expressed as a Wishart projection.
Next, the authors propose two noisy variants: adding Gaussian noise after or before projection. They show that the randomness in the projection can yield stronger DP guarantees than conventional Gaussian mechanism, in the large-rank and small-rank regimes, respectively.
Empirically, the paper shows near-perfect membership inference attacks against noise-free LoRA-style training, and compares the proposed noisy projection method with DP-SGD, DP-LoRA, and DP-LoRA-FA on classification tasks. Their method achieves accuracy generally comparable to the LoRA-based baselines.
Critical strengths
I find the negative result for matrix-valued Wishart projections technically clean. The proof identifies a simple structural obstruction to DP. This helps clarify why privacy guarantees for Gaussian sketches do not necessarily extend to Wishart-projected updates, as might be inferred from prior work such as Malekmohammadi and Farnadi.
(As a side note, the discussion of Malekmohammadi and Farnadi (2024) in Section 5 appears to refer to a later version of the same paper (February 2026), which explicitly claims formal DP guarantees. It may be helpful to update the citation to reflect this version.)
Critical weaknesses
-
I have two concerns about the correctness of the positive privacy results:
- In Algorithm 2, the authors clip gradients with respect to the LoRA factor , which depend on the randomly sampled projection matrix . However, Theorem 5 assumes that the input to the projection mechanism is independent of the projection randomness. Thus, Theorem 5 does not directly justify the privacy accounting used for Algorithm 2 and the corresponding experiments.
- In Lemma 16 (used to prove Theorem 4), the authors apply the classical Gaussian mechanism bound without requiring , even though this bound is not generally valid for larger . This appears to invalidate the claimed privacy guarantee in Theorem 4 for some parameter regimes.
(To be transparent, I used an LLM to identify potential technical issues and then went through them myself. I think both issues are valid and somewhat important to the correctness of the paper's results.)
-
I think the negative result is less significant than what the abstract and introduction seem to suggest. In particular, the paper makes strong claims such as "LoRA is not inherently private" and "the consequence for LoRA is immediate: without additional additive noise, neighbouring datasets can yield non-overlapping support, ruling out any (finite-, finite-) DP guarantee."
However, LoRA usually refers to a fine-tuning procedure involving multiple optimization steps. The paper only establishes that the first update of standard LoRA, where and the gradient is independent of the randomly initialized , can be expressed as a Wishart projection and is therefore not DP in the worst case. In subsequent steps, both and evolve, and the gradients may depend on the adapter parameters from previous steps, so the same argument no longer directly applies. More importantly, in a standard use case, only the final model is released, rather than the intermediate weights . As the authors acknowledge in Section 3.2, their impossibility result does not directly apply to this setting. This limits the significance of the result for practical LoRA training, which is my main reservation about the paper's contribution to an ICLR audience.
-
A minor point: Regarding M2, the authors show that the randomness in the projection can yield stronger DP guarantees than the standard Gaussian mechanism. However, I'm not sure how meaningful this comparison is in terms of utility, since M2 introduces additional approximation error through projection, and thus may have substantially different use case from the standard Gaussian mechanism.
Questions for authors
See the Weakness section.
5 comments
Thank you again for your careful review. As the discussion period ends in two days (November 18 at 11:59pm AoE), we would like to gently remind you of our response above.
We hope our response has clarified the intention behind Algorithm 2 and Lemma 16, as well as the scope of our negative result. We will revise these parts in the paper to make them clearer. If any questions remain, please let us know and we will be happy to discuss further.
Thank you for the detailed clarifications regarding Algorithm 2/Theorem 5 and Lemma 16/Theorem 4. These address my main soundness concerns. Thank you also for explaining the citation to the earlier version of Malekmohammadi and Farnadi.
I also appreciate the additional multi-step counterexample. I agree that it establishes that releasing only the final LoRA model can fail DP in the worst case, and therefore addresses part of my original concern about the scope of the negative result.
My remaining reservation concerns the practical scope of the contribution. Given the paper’s motivation through LoRA, I remain less convinced that the Wishart analysis provides sufficient insight into model privacy in standard multi-step LoRA (beyond the new example ruling out an unconditional privacy guarantee). I understand that a general characterization is beyond the scope of this paper; my concern is whether the established result is sufficiently significant for ICLR. I therefore lean toward retaining my score.
Thank you for your careful reading. Let us first address your concerns about the correctness of the privacy guarantees in Theorems 4 and 5.
Algorithm 2 and Theorem 5. The reference to Algorithm 2 in the experimental section was an oversight. This issue was also pointed out by another reviewer and we apologise for this mistake. The experiments evaluating Theorem 5 implement M2: they compute and clip the per-example full-weight gradients, add Gaussian noise, and then apply a fresh projection At. The clipped input is therefore independent of At, as Theorem 5 requires.
We only included Algorithm 2 to illustrate how the projected update can be computed directly in LoRA coordinates without materializing full-weight gradients when clipping is inactive. With active clipping in those coordinates, the multiplier depends on At even though the threshold is fixed. We will clarify this distinction and correct the experimental cross-reference.
Lemma 16 and Theorem 4. We presented the classical Gaussian calibration because it gives a simple closed-form bound for 0<ε∥<1. The same argument extends to any ε∥>0 using the analytic Gaussian calibration of Balle and Wang 2018, which requires solving a scalar equation numerically and is therefore less compact to present. Our experiments already use the analytic calibration. We will add a remark clarifying both the extension and the calibration used in our experiments.
Scope of the negative result. Proposition 2 shows that the first noise-free update can violate DP, ruling out privacy for any released transcript that includes it. We will make that scope explicit in the abstract and introduction. However, we want to highlight that standard privacy accounting for DP-LoRA and DP-SGD composes guarantees for individual updates and the first-step result therefore shows why standard composition cannot certify its privacy.
To our knowledge, there is no general privacy characterization for releasing only the final model from standard LoRA. Establishing such a characterization across loss functions would be a substantial research contribution in its own right and is beyond the scope of this paper. However, a simple worst-case example showing that the support separation in Proposition 2 can persist through every training step is the following: Consider neighboring one-record datasets with losses ℓ0(W)=0 and ℓ1(W)=W11. The zero loss leaves WT=W0. For ℓ1, let at=Ate1 and bt=Bt⊤e1. Gradient descent gives at+1=at−ηbt,bt+1=bt−ηat. Since b0=0 and a0=0 almost surely, at and bt are nonzero and point in opposite directions for all t≥1. Thus (WT−W0)11=bT⊤aT<0 almost surely under ℓ1. This event has probability 0 under ℓ0 and 1 under ℓ1, so releasing only WT violates DP for any δ<1.
For a more realistic setting we assessed final-model leakage empirically on CIFAR-10 (see Table 1 in the paper). A membership inference attack using only the final model achieves ROC-AUC above 0.99, showing that membership is effectively exposed for the tested examples even when all intermediate updates are withheld.
Related work (Malekmohammadi and Farnadi): We cited Malekmohammadi and Farnadi's 2024 preprint because it motivated our study of LoRA without additive noise. We omitted the February 2026 revision because we believe its privacy proof has a gap. It bounds privacy for a sketch of the form Z⊤V, then reuses the same random Z to reconstruct the update. The privacy bound for the sketch alone does not justify that step. Our impossibility result gives a counterexample for the noise-free update ZZ⊤V. We will cite both versions and explain the distinction.
We hope these clarifications address your concerns about the privacy guarantees and show why the negative result matters in practice, even when only the final model is released.