FedSpec: Shared Spectral Structures for Heterogeneous Federated Learning

Model heterogeneity poses a fundamental challenge to federated learning, as different model architectures produce incompatible parameter and representation spaces. Existing heterogeneous federated learning methods mainly rely on parameter sharing or feature alignment, often requiring compatible model components or explicit alignment mechanisms. To address this issue, we propose FedSpec, a heterogeneous Federated learning framework based on shared Spectral structures. FedSpec transforms heterogeneous local feature spaces into fixed-length normalized spectral structures through singular value decomposition, aggregates them into a global spectral structure, and uses the global spectrum as a structural prior for local spectral correction. The singular values characterize the relative importance distribution of principal representation directions, enabling a unified structural representation across heterogeneous models without requiring parameter compatibility or explicit feature alignment. We further analyze the convergence behavior and information reduction properties of FedSpec. Extensive experiments across diverse architectures and data modalities demonstrate that FedSpec consistently outperforms state-of-the-art methods, achieving up to 7.67% accuracy improvement with substantially lower communication cost.