Final-Model Privacy for Proximal-Anchored DP-SGD
Abstract
Standard R\'enyi differential privacy (RDP) composition charges a record at every access, so its bound grows with the number of fine-tuning updates even when only the final model is released. We study this hidden-state setting for clipped, projected proximal-anchored DP-SGD. Our shifted-R\'enyi PABI certificate separates the discrepancy created by an accessed record from the state difference propagated by later updates. Exact Gaussian smoothing gives dimension-free propagation factors, while a bounded correction at a fixed public reference state reduces the smoothing bias there without changing those factors. For any public access pattern, causal shift allocation computes the certificate in time. If the certified factors contract on every contiguous interval, the final-model RDP bound is independent of and has no separate domain-diameter term. We also give an adaptive anchor rule with a joint bound for its privatized monitor bits and final model. At 500 epochs, the privacy loss is , compared with under step-wise composition.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.