Rank-Dependent DP-Noise Bounds for Frozen-$A$ LoRA

We study differentially private SGD (DP-SGD) for frozen-$A$ LoRA, where a random factor $A\in {\mathbb{R}}^{r\times n}$ is fixed and only $B\in {\mathbb{R}}^{m\times r}$ is trained. Privacy accounting is dimension independent: the same noise multiplier, Poisson sampling rate, step count, adjacency convention, and accountant yield the same $(\epsilon ,\delta )$ guarantee. Our main result is instead a utility statement for the restricted $B$-objective: when $A_{jk}\sim \mathcal{N}(0,1/n)$ gives $\Vert A{\Vert }_{\mathrm{op}}\le \kappa$, the reparameterization controls both gradient sensitivity and smoothness, and the DP-noise component of the stationarity bound scales with $mr$ rather than $mn$. With inactive clipping, $C_L=\kappa C_F$, and respective positive initial gaps ${\Delta }_D,{\Delta }_F$, the ratio of the DP-noise terms is bounded by ${\kappa }^2\sqrt{r{\Delta }_D/(n{\Delta }_F)}$. With active clipping, method-specific bias terms remain and we do not claim an unconditional improvement in total error or practical utility. The guarantee controls ${\nabla }_B{L}_D$, not unrestricted stationarity in $W$-space. A Polyak–Lojasiewicz (PL) assumption is used only for excess-risk and optimal-rank results; we prove it for linear LoRA, while the informed-$A$ optimal-rank result is an oracle benchmark. We quantify, without a complete convergence theorem, the product-noise term arising when both factors are trained. A complete 1,800-run controlled synthetic matrix separates restricted error from projection error and demonstrates optimizer-dependent rank orderings; SST-2 experiments remain feasibility checks, not validation of nonlinear PL or the optimal-rank law. A completed 42-run matched-public-start BERT/SST-2 follow-up finds at most 0.153 percentage points mean accuracy difference between frozen and jointly trained factors across six private-stage settings, providing no clear practical advantage for either branch.