FedSALT: Subspace-Drift-Guided Adaptive Local Training for Federated LoRA Fine-Tuning

Federated low-rank adaptation (LoRA) enables efficient fine-tuning of large language models (LLMs) over decentralized data by communicating only compact low-rank adapters. However, heterogeneous client tasks naturally produce task-specific LoRA updates, making it difficult to learn a single global model through aggregation. We observe an empirical asymmetry in this heterogeneity: task differentiation is substantially more pronounced in the output-side geometry of LoRA updates than in the input-side geometry. Motivated by this observation, we propose FedSALT, a geometry-aware closed-loop optimization framework that maintains a momentum-based consensus subspace, measures client-module disagreement using a factorization-invariant output-subspace score computed directly from the realized LoRA update, and converts the score into adaptive proximal regularization for subsequent local optimization. Unlike existing methods that apply uniform regularization or intervene only during aggregation, FedSALT selectively guides drifting modules toward a momentum-lookahead consensus anchor while retaining standard factor-wise FedAvg aggregation. Experiments on three heterogeneous federated LLM benchmarks show that FedSALT consistently achieves the best overall accuracy among representative baselines. Mechanism studies further validate the motivating output-side asymmetry and demonstrate the effectiveness of adaptive geometry-aware control.