Distributed Nonconvex Learning under Coupling Constraints over Directed Networks

Distributed learning is often formulated through consensus, yet many collaborative models are built from distinct local components that must satisfy global consistency relations rather than agree with one another. This becomes particularly challenging when the objectives are nonconvex, gradients are stochastic, and communication is directed and unbalanced, since global feasibility evolves with the local models and cannot be evaluated at any single node. We study this setting through general affine coupling and propose DiSPART (Directed Stochastic Primal–dual Algorithm with Residual Tracking). DiSPART turns global constraint information into distributed feedback by combining push–sum residual and multiplier tracking with gradient smoothing and moving-center regularization. Under standard smoothness, constraint regularity, bounded-variance stochastic gradients, and an explicit Perron-weighted mixing condition, we establish a finite-time bound on the average expected squared KKT residual that jointly controls stationarity and feasibility. For fixed problem and network parameters, the bound gives $\mathcal{O}(K^{-1/2})$ convergence with stochastic gradients and $\mathcal{O}(K^{-1})$ with exact gradients, while allowing infeasible primal initialization and a horizon-independent penalty without assuming bounded multipliers. The analysis further makes explicit how affine coupling, network mixing, and Perron imbalance shape finite-time performance. Experiments support the algorithmic design and the predicted network effects, while two real-data classification tasks show that these guarantees translate into practical constrained learning with substantially lower gradient-query costs than exact-gradient baselines.