FedPolicy: An RL-Guided Redistribution Policy for Synergizing Local-Global Optimization in Federated Learning

Statistical heterogeneity remains a central challenge in federated learning, where uniform post-aggregation broadcast can overwrite client-specific structure and cause negative transfer. Existing approaches primarily address heterogeneity through local objectives, server aggregation, or personalization, while clients typically receive the same parameter block after aggregation. We propose FedPolicy, an RL-guided post-aggregation redistribution framework that treats the return path of the aggregated model as a client-specific decision problem. Rather than broadcasting the same update to every client, FedPolicy learns which part of the aggregated model should be transferred back to each client by selecting among full-model, backbone-only, and head-only parameter blocks. This formulation identifies post-aggregation redistribution as a previously underexplored control axis in federated optimization, improving the balance between global transfer and local specialization. Extensive experiments under heterogeneous federated settings show that FedPolicy consistently outperforms strong baselines across CIFAR-10, CIFAR-100, and ISIC2019, with the clearest gains appearing in the more challenging heterogeneous regimes. Across the settings in the main comparison, FedPolicy achieves an average relative gain of approximately $7.30\%$ over the strongest baseline, with the largest improvement reaching $16.47\%$ on ISIC2019 under severe heterogeneity; the supplementary FMNIST and UCI HAR study shows the same direction of improvement. It also converges faster and improves the cost-to-accuracy trade-off, while adding only 0.021 seconds of controller computation per round. These results identify client-specific post-aggregation redistribution as an effective and underexplored design dimension in heterogeneous federated learning.