DP-RoLL: Fortifying Private Low-Rank Learning against Data Contamination

Multivariate response learning often involves sensitive individual-level data and corrupted observations. Low-rank coefficient structure can improve statistical efficiency under differential privacy by exploiting shared response factors, but its benefits may be compromised by outliers. In this work, we propose a differentially private and robust low-rank learning framework for a broad range of loss functions. Theoretically, we establish a non-asymptotic error bound that decomposes the error into optimization, statistical, and privacy-induced components and reveals an optimization-privacy trade-off in the number of private iterations. Extensive experiments demonstrate the effectiveness of our proposed framework in protecting sensitive data while maintaining strong performance in the presence of data anomalies.