Geometry-Aware Subspace Perturbation for Heterogeneous Federated Learning

We propose a geometry-aware perturbation framework that explicitly models the low-rank, anisotropic, and evolving structure of gradient dynamics to improve generalization in heterogeneous federated learning (HFL). Our approach is built upon three key components. First, we represent perturbations within a low-dimensional subspace that captures the dominant directions of gradient trajectories. Second, we generate perturbations through a structure-aware sampling strategy that aligns with the covariance of projected gradients, enabling distribution-aware exploration. Third, we introduce an adaptive mechanism that dynamically adjusts both the magnitude and the structure of perturbations to match the evolving optimization process. Extensive experiments demonstrate that the proposed method consistently improves generalization across diverse HFL benchmarks, achieving up to a $4.49\%$ improvement on the $\text{Office10}$ dataset with minimal computational overhead.