FedRespro: Breaking Resource-Inverse Distribution Dilemma via Federated Residual Projection

Heterogeneity has long been a fundamental obstacle to effective model collabora- tion in federated learning (FL). Despite extensive efforts to address data, system, and model heterogeneity, existing methods remain inadequate for a prevalent yet largely overlooked scenario, which we term Resource Inverse Distribution (RID), where data-rich frontline clients are constrained by limited model capacity, while high-capacity clients have access to only limited data. RID disperses statistical reliability and optimization capacity across different clients, inducing Statistical-Optimization Capability Decoupling: low-capacity clients cannot sufficiently ex-plore effective directions beyond their constrained optimization spaces, whereas complementary knowledge discovered by high-capacity clients suffers from sta-tistical uncertainty due to limited observations. To address this dilemma, we pro-pose FedRespro, a federated heterogeneous model collaboration framework that recouples statistical reliability and optimization capacity across heterogeneous clients. Specifically, FedRespro extracts complementary residual knowledge from the expanded optimization spaces of high-capacity clients and suppresses unre-liable information through statistical, spatial, and temporal reliability estimation. It further constructs rank-matched references via capacity-aware low-rank projec-tion, enabling data-rich low-capacity clients to absorb complementary knowledge beyond their local exploration capabilities. By compensating for the statistical unreliability of high-capacity clients and the constrained optimization capabil-ity of data-rich clients, FedRespro enables effective cross-client complementar-ity between statistical and optimization advantages under RID scenarios. Experi-ments show an average server accuracy gain of 2.19 percentage points across three Qwen2 scales over the strongest evaluated baselines. On BERT-base, CoLA MCC (×100) improves by 23.43 points, while DBpedia accuracy improves by 16.81 percentage points, each relative to its strongest evaluated baseline. A reference implementation is available at https://anonymous.4open.science/r/ FedRespro/.