An Adaptive Penalty Method for Heterogeneous Federated Learning with Nonlinear Constraints

Federated learning (FL) enables collaborative model training without sharing raw data and is typically formulated as unconstrained minimization of a global loss. To the best of our knowledge, no existing methods provide provable convergence guarantees for nonlinearly constrained FL while explicitly mitigating client drift. We propose an adaptive-penalty Moreau-smoothed stochastic controlled federated averaging method that extends SCAFFOLD to FL with both local and global nonlinear constraints. The penalty parameter and stepsize are updated at each communication round according to analytically derived rules. In the convex setting, we achieve $\tilde{O}({ϵ}^{-2})$ communication complexity for both global suboptimality and constraint violation. In the strongly convex setting, the complexity for both quantities improves to $\tilde{O}({ϵ}^{-1})$. Some of the unique features of our method include the following: (i) In both regimes, the global suboptimality complexity matches the $ϵ$-dependence of SCAFFOLD and FedAvg in the unconstrained setting; (ii) unlike limited existing constrained FL analyses that rely on Lagrangian duality and constraint qualifications, our penalty-based analysis establishes upper bounds on both global suboptimality and constraint violation without requiring constraint qualifications. We show that a constraint qualification is needed only to establish the lower bound on global suboptimality; and (iii) under Slater's condition, we show that the expected suboptimality and infeasibility errors converge asymptotically to zero for any arbitrary Moreau smoothing parameter. Numerical experiments on MNIST, EMNIST, and CIFAR-10 show that the proposed method attains competitive performance while consistently reducing constraint violation.