Consistent Subspace Aggregation for Personalized Federated LoRA

Federated fine-tuning with LoRA suffers from an aggregation inconsistency, as averaging low-rank factors does not recover the average of the updates they induce. Existing remedies need both factors at the server, but personalized protocols share only the right factors and keep client-specific left factors local, where factor averaging depends on each client's arbitrary factorization. We propose PERSIA, a fully low-rank aggregation rule for this setting, where each client whitens its right factor with the Gram matrix of its private left factor, and the server recovers a shared right subspace by truncated SVD. We prove that the core PERSIA rule exactly minimizes the clients' reconstruction error among all shared rank-$r$ right factors, with an objective invariant to equivalent factorizations, and give a factor-stationarity bound whose floor does not grow with the number of local steps. On GLUE with RoBERTa-Large, right after aggregating a 1280-step local phase, PERSIA's personalized models are as accurate as the clients' local models (91.1% versus 90.8%), whereas factor averaging loses 4.1 points and two-factor FedAvg 11.3 points; after further rounds, final accuracy is on par across rules, with PERSIA uploading half as much as two-factor sharing.