Conflict-Averse Update Decoupling for Federated Multi-Task Learning

Federated Multi-Task Learning (FMTL) enables clients with different tasks to collaboratively train models without sharing raw data. However, task and data heterogeneity jointly induce conflicting updates across clients. Existing FMTL methods treat the aggregated update each client receives from the server as a whole, or keep only the dimensions selected by magnitude; such updates may oppose the client's local direction, so the two cancel each other out and cause negative transfer. In this paper, we propose CAUD (Conflict-Averse Update Decoupling), a personalized aggregation method that mitigates parameter conflicts in FMTL without introducing any additional learnable parameters or network modules. Specifically, we adopt a vector obtained from a conflict-averse max-min problem as the global guidance, and decouple each client's update into consensus and client-specific parts through element-wise direction consistency with this guidance; the two parts are then modulated with closed-form weights to produce a personalized update for each client, thereby suppressing conflicts while retaining beneficial task-specific knowledge. Extensive experiments on the standard PASCAL-Context and NYUD-v2 benchmarks demonstrate that CAUD consistently improves performance across various heterogeneous client configurations.