FedConsult: Turning Client Heterogeneity into On-Demand Expertise for Personalized Federated Learning

Federated learning enables decentralized clients to collaboratively train models without sharing private data. Existing methods combine collaborative knowledge sharing with local adaptation to improve predictions under statistical heterogeneity, yet personalization gains remain client-bound, leaving substantial performance gaps across clients on specific classes. To address these limitations, we propose FedConsult, a federated expertise-transfer framework that distills locally acquired strengths into reusable visual specialty adapters. These adapters let each client complement its predictions with others' expertise on a sample-by-sample basis, improving weak-class recognition while preserving local personalized strengths. Specifically, FedConsult identifies each client's regions of specialty competence from discrepancies between its global and personalized prediction paths, and distills visual knowledge with similar prediction-improvement patterns into specialty adapters in a shared feature space. Recipient clients subsequently use response-aware sparse routing to selectively invoke complementary expertise within their personalized pathways. Experiments on both natural and medical image classification tasks show that FedConsult outperforms the strongest baselines by up to 3.96 and 9.13 percentage points in overall and client-specific weak-class accuracy, respectively. Beyond viewing heterogeneity solely as an optimization challenge, FedConsult turns the resulting capability diversity into transferable expertise, closing local capability gaps and supporting more reliable personalized federated systems.