FedRRA: Residual Rank Adaptation for Heterogeneous Federated Fine-Tuning

Combining Large Language Models (LLMs) with Low-Rank Adaptation (LoRA) and Federated Learning (FL) has emerged as a promising paradigm to enhance Parameter-Efficient Fine-Tuning (PEFT) while preserving data privacy. However, in practical deployments, where clients have heterogeneous resources and thus require different LoRA ranks for fine-tuning, the conventional paradigm of aggregating uniform LoRA adapters is no longer applicable, introducing significant challenges. While padding-based methods, such as zero-padding, have been proposed to address this issue, we find that they still suffer from two key challenges: limited adaptability to heterogeneous resource settings and aggregation noise. To address these challenges, we propose Federated Residual Rank Adaptation (FedRRA), a novel heterogeneous Federated Fine-Tuning (FFT) framework. Specifically, we design a new padding strategy, termed Consensus-Aware Padding (CAP), which performs weighted padding by combining rank-one signal strength with direction-aware soft gating, enabling improved adaptation to diverse resource-heterogeneous scenarios. In addition, we propose a residual correction method tailored for heterogeneous resource settings, which mitigates aggregation noise by adding a residual term to the frozen pretrained weights. Extensive experiments demonstrate that FedRRA consistently outperforms existing methods across different resource-heterogeneous settings.