Model-Agnostic Projection Optimization for Communication-Efficient Federated Learning

Federated learning enables collaborative model training across distributed clients without sharing sensitive data. However, communication overhead remains a significant bottleneck. Low-rank decomposition techniques address this by approximating each layer’s weights or gradients as a product of low-rank matrices, thereby reducing communication costs in FL. While effective, these methods are typically layer-wise and constrained by the layer's architecture and shapes, limiting their flexibility and performance. We propose **Model-Agnostic Projection Optimization** (MAPO), a novel model-wide low-rank parametrization method that factorizes the full model gradient into a *fixed reconstruction matrix* and a *trainable projection vector*, thereby avoiding layer-wise decomposition and architecture constraints. MAPO directly optimizes the projection in a randomly sampled subspace, with all clients generating the reconstruction matrix via a shared random seed, incurring no additional communication overhead. By decoupling the gradient from architecture through reshaping and enabling communication-free exploration of subspaces via seed sharing, MAPO provides a flexible and efficient low-rank representation. Across seven FL benchmarks, MAPO reaches 92–99.7% of FedAvg's accuracy while using 0.13–3.1% of its uplink, exceeding all evaluated baselines.