The Wishart Projection Mechanism Is Almost Differentially Private: Guarantees and Limitations

We introduce and analyse the *Wishart projection mechanism*, a randomised map of the form ${\mathcal{A}}_r(S):=f(S)M$ where $M\sim {\mathsf{W}}_d(\frac{1}{r}{I}_d,r)$, and study its differential privacy (DP) properties. For vector-valued queries $f$, we prove non-asymptotic DP guarantees showing that Wishart randomness alone suffices, without requiring any additive noise. For matrix-valued queries, however, we establish a sharp separation: we prove that without additive noise the mechanism is not DP, and empirically supplement this theoretical result with a near-perfect membership inference attack ($\mathrm{AUC}>0.99$). As a concrete instantiation, we show that LoRA-style updates are an instance of the matrix-valued Wishart projection mechanism, implying that LoRA is not inherently private despite its initialisation randomness, and that empirical observations of reduced memorisation under LoRA should not be interpreted as DP guarantees. We then analyse a noisy variant and prove that the Wishart projection can *amplify* the resulting privacy guarantee: at a fixed noise multiplier, composing with a freshly resampled projection yields strictly stronger DP bounds than noise alone, both in the large-$r$ and small-$r$ regimes.