LoRA-based Data Augmentation for Federated Learning on Non-IID Data

Federated learning (FL) enables privacy-preserving collaborative training. However, its performance degrades substantially under non-independent and identically distributed (non-IID) client data due to client drift or distribution skew. Existing methods often fail to provide distribution-aligned samples to local clients. We propose Federated Harmonization via Diffusion with Low-Rank Adaptation, Federated Harmonization via Diffusion with LoRA, or simply FedHDL, a generative framework that leverages a pretrained Stable Diffusion backbone with lightweight local LoRA adapters. Instead of aggregating full parameters, the server aggregates only LoRA adapters and performs cross-style synthesis to generate missing class–style samples. By using these synthetic samples for data augmentation, FedHDL effectively aligns local data with the global distribution, thereby re-balances label distributions and mitigates feature skew while maintaining low communication overhead. Extensive experiments across diverse non-IID scenarios show that FedHDL consistently improves upon prior generative augmentation methods while achieving competitive performance against strong federated optimization and personalization baselines. The experimental results demonstrate that parameter-efficient generative alignment is an effective path toward robust federated learning under distribution shift.