Decoupling Federated Learning to address Utility-driven asynchrony

Federated learning (FL) enables scalable collaborative training through a server that establishes a fixed communication structure and governs the aggregation of parameters sent by the clients. Efficient FL is marred by exogenous factors such as stragglers, network delays, and staleness, and endogenous factors such as local distribution shifts. These factors introduce arbitrary client availability and changes in the aggregation mechanism – unobservable to the server. In fact, the effect of such phenomenon is only observed in the clients' utility. Clients must be allowed to drive learning according to their local utility. This does not happen at present because client and the server depend on each other for updates leading to a coupling completely driven by the server. In this paper, we enable utility-driven FL, where the clients can communicate based on “how their local utility changes with time?” instead of a pre-determined communication structure. We leverage dynamic-programming to develop a novel framework that decouples client-server communication dependency and design \uafl where client can drive learning based on their own utility. Empirically and theoretically, we show that convergence can be achieved in spite of no predetermined communication structure and in the presence of staleness, client churn, gradient magnitude variance, and angular misalignment. In particular, without prior knowledge of the endogenous/exogenous factors present in the dataset, \uafl outperforms standard methods, carefully designed for specific (endogenous/exogenous) factors on standard benchmarks and realistic datasets by upto $46\%$.