- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Paper summary The paper studies federated multi-task learning (FMTL) under task and data heterogeneity, where different clients may pursue different tasks. Such heterogeneity causes their parameter updates to have conflicting directions at the individual parameter-coordinate level. Conventional aggregation can therefore cancel useful updates and lead to negative transfer. The paper proposes Conflict-Averse Update Decoupling (CAUD) to distinguish shared, consensus information from client-specific information before aggregation.
Contributions:
- CAUD constructs a max-min global guidance vector within a trust region around the mean client update. The vector is designed to be compatible with all clients rather than simply averaging their potentially conflicting updates.
- Each client update is decomposed coordinate-wise according to whether its direction agrees with the global guidance. Consistent coordinates are treated as consensus knowledge, while conflicting coordinates are retained as client-specific knowledge.
- Consensus coordinates are guided by the global direction, whereas client-specific coordinates are continuously modulated according to their relative magnitude instead of being simply discarded. The resulting personalized updates require no additional learnable parameters or modules.
Validation:
The method is evaluated on PASCAL-Context and NYUD-v2 under heterogeneous FMTL settings and compared with a broad range of federated and multi-task aggregation methods. CAUD achieves a 4.18% Δm, compared with 2.18% for the strongest baseline, FedHCA2, in the main heterogeneous setting.
Critical strengths
- The key strength is the explicit focus on parameter-level update conflicts rather than treating each client update as an indivisible vector. By decoupling consensus and client-specific coordinates and handling them differently, CAUD provides a more fine-grained way to balance shared and task-specific knowledge. This is consequential because parameter conflict is a fundamental challenge in heterogeneous FMTL, and the proposed formulation directly targets the mechanism that can cause update cancellation and negative transfer.
- CAUD achieves a substantially higher overall improvement than the strongest baseline in the main heterogeneous setting (Δm of 4.18% versus 2.18% for FedHCA2), while ablations show progressive gains from conflict-averse guidance, coordinate-level decoupling, and adaptive modulation. The parameter-level analysis further supports the claim that CAUD produces more coordinated cross-client updates. This combination of comparative, ablation, and mechanistic evidence makes the empirical conclusions more convincing than relying on overall performance alone.
Critical weaknesses
- The proposed global guidance vector is explicitly formulated as the update-space analogue of CAGrad, and the paper acknowledges that the same max-min conflict-averse formulation has already been federated in prior work. The main additional contribution is therefore the subsequent sign-based coordinate decoupling and scalar modulation. However, the decoupling itself relies on a relatively simple heuristic, which is that the coordinates whose update signs agree with the global guidance are labeled as consensus, while the remaining coordinates are treated as client-specific. This binary sign criterion does not establish whether a conflicting coordinate is genuinely task-specific or merely an undesirable optimization conflict. Consequently, the conceptual gap between CAUD and prior conflict-averse aggregation/personalization methods is not sufficiently substantial to support the paper's broad claim of a new solution to parameter-level conflict in FMTL.
- The central design of CAUD depends on interpreting directionally consistent coordinates as shared knowledge and conflicting coordinates as client-specific knowledge. Yet the experimental evidence mainly demonstrates that the resulting update transformation improves task performance and increases cross-client update similarity, neither directly verifies the semantic or functional meaning of the two coordinate groups. In fact, the ablation shows that simply discarding the client-specific coordinates (3.86%) performs better than retaining them without modulation (3.60%), while the proposed adaptive modulation provides only a further 0.32-point gain (4.18%). This makes it difficult to establish that the conflicting coordinates actually contain useful task-specific information, rather than being predominantly harmful interference that can simply be suppressed. Since this assumption is fundamental to the claimed “consensus vs. client-specific knowledge” interpretation, the lack of direct validation substantially weakens the methodological justification of CAUD.
Questions for authors
- What is the empirical or theoretical evidence that sign consistency with the global guidance vector reliably identifies “consensus” knowledge, while sign inconsistency corresponds to “client-specific” knowledge?
- Since βᵢ is a single scalar applied uniformly to all client-specific coordinates, how does CAUD distinguish useful task-specific updates from harmful conflicting updates at the coordinate level?