Sporadic Gradient Tracking over Directed Graphs: A Theoretical Perspective on Decentralized Federated Learning

Decentralized Federated Learning (DFL) enables clients with local data to collaborate in a peer-to-peer manner to train a generalized model. In this paper, we propose a novel formulation that unifies two branches of work that have separately solved important challenges in DFL as special cases: (i) push-pull gradient tracking techniques for stable convergence over directed graphs and (ii) resource-aware methods for handling heterogeneous client. We propose *Sporadic Gradient Tracking* ($\text{Spod-GT}$), the first DFL algorithm that incorporates these factors over directed graphs by allowing flexible (i) client-specific gradient computation frequencies and (ii) heterogeneous and asymmetric inter-client communication frequencies. We conduct a rigorous convergence analysis of our methodology with a relaxed assumption on gradient diversity, providing $\mathcal{O}(1/K)$ consensus and stationarity guarantees for GT over directed graphs under intermittent client participation. Through numerical experiments on image classification datasets, we demonstrate the efficacy of $\text{Spod-GT}$ compared to well-known GT baselines.