acceptodds
Withdrawn from ICLR 2027

PathwayFed: Disentangling Client-Specific Knowledge Pathways for Federated Multi-Label Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Federated Multi-Label Learning (FML) enables multiple clients to collaboratively learn from heterogeneous multi-label data while preserving data privacy. However, variations in local label spaces and label co-occurrence patterns cause different clients to encode specialized knowledge through entangled feature-to-class path- ways, resulting in implicit conflicts and the dilution of informative evidence during aggregation. To address this challenge, we propose PathwayFed, a novel frame- work that makes client-specific knowledge pathways more distinguishable and disentangles their contributions during federated aggregation. On the client side, PathwayFed refines knowledge pathways through dimensional expansion and an orthogonal sparse transformation, producing finer-grained and less-overlapping pathway expressions that make meaningful client-specific knowledge easier to identify and quantify. On the server side, we introduce a matched knowledge routing mechanism that reinforces strongly expressed client-specific knowledge while preserving its pathway association, followed by conflict-aware projection to suppress conflicting components and calibrate the compatibility of the routed knowledge pathways. Extensive experiments on PASCAL VOC 2007, MS-COCO 2014, and NUS-WIDE-81 demonstrate that PathwayFed outperforms state-of-the- art methods, particularly under severe heterogeneity.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Federated Learning, Multi-Label Learning, Client-Specific Knowledge Pathways, Conflict-Aware Aggregation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RMWWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new federated multi-label learning framework named PathwayFed, aiming to mitigate the dilution of client-specific contributions during aggregation under heterogeneous local data distributions and label co-occurrence patterns. Its core essence lies in training a multi-label prediction head on frozen CLIP features, then selectively incorporating client contributions based on specific parameters while attenuating updates that conflict in direction. This paper validates the effectiveness of the proposed model through experiments on three visual datasets with varying degrees of heterogeneity, alongside comparisons with seven baselines.

Critical strengths

a. The experimental section is quite comprehensive, covering varying degrees of heterogeneity, ablation study, routing weight shuffling, normalization control, and partial client participation.

b. The pathwayFed framework is well-structured. Clients Clients refine pathway-related representations through expansion and sparse representation, while the server combines parameter-level routing with conflict projection. Furthernore, the overall framework a comprehensive solution addressing the issue where aggregation dilutes the specialized expertise of heterogeneous clients.

Critical weaknesses

a. The advantages of PathwayFed under mild heterogeneity are limited in visual scenarios. The performance improvements on PASCAL VOC 2007 and MS-COCO 2014 under mild heterogeneity are limited, with results sometimes even falling below the strongest baselines on some metrics.

b. The justification of the key routing mechanism is insufficient. There remains an explanatory gap between the use of update magnitude as a proxy for knowledge contribution and the retention of useful client-specific knowledge emphasized in the paper.

Questions for authors

a. The captions for Table 1 and Table 8 indicate that the gains are calculated relative to the strongest existing method. However, some figures appear to be based on weaker baselines. Could the authors verify all the gain entries and clarify whether the corrected values ​​affect the discussion of the results?

b. Regarding the primary experimental results in Table 1, the authors should consider specifying the conditions under which the advantage emerges or defining the scope of applicability.

c. The values ​​in Table 3 (main text) and Table 12 (appendix) differ, even though both use the COCO (K=80) setting. Is there a difference in the configuration? Several other tables and figures exhibit similar issues.

d. Can the authors clarify the relationship between the activation-based pathway contribution in Eq. (7) and the update-based routing scores in Eqs. (10)–(12)? It would be best to provide a brief theoretical analysis of the entire PathwayFed model.

Reviewer_iWNROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PathwayFed for federated multi-label learning, aiming to mitigate conflicts and the dilution of client-specific knowledge during aggregation under heterogeneous label spaces and label co-occurrence patterns. Using frozen CLIP features, the method combines dimensional expansion and a fixed orthogonal sparse transformation to refine pathway representations. On the server, matched knowledge routing assigns component-wise aggregation weights from local update magnitudes while preserving client–pathway correspondence, followed by conflict-aware projection to suppress incompatible update components. Evaluation on PASCAL VOC 2007, MS-COCO 2014, and NUS-WIDE-81 reports competitive performance, with larger advantages under stronger heterogeneity. Component ablations, routing normalization and permutation controls, and pathway concentration diagnostics examine the contributions and behavior of the proposed mechanisms.

Critical strengths
  1. A concrete aggregation design for the evaluated multi-label setting. For multi-label classification with frozen CLIP features, PathwayFed combines matched knowledge routing with conflict-aware projection to address the dilution and interference of heterogeneous client contributions. The server computes routing weights from ordinary local head updates without requiring additional activation or label statistics. This is a practically relevant design choice within the evaluated setting, although its applicability to other backbones or tasks remains untested.

  2. Targeted controls that strengthen the empirical investigation. The paper includes routing-weight permutation to examine client–pathway correspondence, client-scale and sample-count normalization to assess sensitivity to overall client update magnitude and local data volume, and fixed-capacity semantic mixing to vary specialization while preserving client counts and sizes. These controls are a substantive strength of the experimental design because they address specific alternative explanations for the reported gains.

Critical weaknesses
  1. Unclear provenance and comparability of baseline results. All six baselines other than FedHarmony in Table 1, together with the five additional baselines in Appendix Table 8, have results identical to the corresponding entries in Table 1 of FedMPT [1]. However, the manuscript does not clarify whether these results were independently reproduced or taken from [1], or whether their evaluation settings are consistent with those used for PathwayFed. This ambiguity makes it difficult to assess the fairness of the comparisons and the strength of the reported improvements.

  2. The reported performance gains do not follow a clear and consistent reference rule. Table 1 describes its Δ rows as differences from the “strongest prior method in the full comparison,” but does not specify whether this means selecting one reference method for each experimental setting or selecting the strongest baseline separately for each metric. These interpretations lead to different conclusions. For COCO at t=100%, FedHarmony has the highest baseline mAP, and comparing CF1 against this method gives the reported gain of 12.30 (63.64−51.34). However, the strongest baseline for CF1 is FedMPT, yielding a gain of only 6.68 (63.64−56.96). A fixed-method interpretation does not consistently explain the reported values either. For NUS-WIDE at t=20%, the mAP gain of 1.07 is calculated against FedHarmony (60.40−59.33), whereas the CF1 gain of 8.74 is calculated against FedMPT (58.03−49.29), despite FedHarmony having the higher CF1 of 50.27. The CF1 gain against FedHarmony would be 7.76. Under the conventional per-column best-baseline interpretation, I find ten discrepancies, all overstating the gains. Although correcting these entries does not eliminate the reported average performance advantage, it materially changes its magnitude. The authors should define the reference rule explicitly and recompute the Δ rows accordingly.

[1] Xucong Wang et al. “FedMPT: Federated Multi-Label Prompt Tuning of Vision-Language Models.” CVPR, 2026.

Questions for authors
  1. Baseline provenance and evaluation consistency. The baseline results in Table 1 and Appendix Table 8 match the corresponding results reported in FedMPT [1]. Could the authors explain whether the data partitions, training budgets, and evaluation procedures underlying these results are consistent with those used for PathwayFed? This clarification is important for assessing the fairness of the comparisons.

  2. Definition of the reported gains. Could the authors specify how the “Δ vs. full prior best” rows in Table 1 are calculated? Is one reference method selected for each experimental setting, or is the strongest baseline selected separately for each metric? For example, on NUS-WIDE at t=20%, the reported mAP gain uses FedHarmony as the reference, whereas the CF1 gain uses FedMPT, despite FedHarmony having the higher CF1. Clarifying the rule and checking the corresponding entries would help establish the magnitude of the improvements.

  3. Conflict projection and specialized client contributions. Table 2 shows an overall benefit from conflict-aware projection. Could the authors nevertheless discuss whether an update opposing the aggregate reference direction might reflect useful knowledge from a client with a substantially different data distribution? How does the proposed mechanism balance suppressing incompatible updates with retaining such specialized contributions? Any available category-level analysis would help clarify this potential trade-off.

  4. Generalizability beyond the current feature extractor. PathwayFed has potential practical value for federated multi-label learning. Could the authors provide a limited additional experiment, such as evaluating the method with a different frozen feature extractor on an existing benchmark, to assess whether its benefits generalize beyond the current CLIP ViT-B/16 setting? Such evidence would strengthen the case for its broader applicability.

  5. Verification of bibliographic information. Could the authors check the reference entries for “Federated Optimization in Heterogeneous Networks” and “An Element-Wise Weights Aggregation Method for Federated Learning”? The former appears to have an incomplete author list, while the page information for the latter may be inaccurate. Please verify these entries against the publication records and provide the complete, corrected information.

[1] Xucong Wang et al. “FedMPT: Federated Multi-Label Prompt Tuning of Vision-Language Models.” CVPR, 2026.

Reviewer_rvxBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces PathwayFed for heterogeneous federated multi-label learning. It captures client-specific knowledge through feature-to-class "pathways," refines them with dimensional expansion and an orthogonal sparse transformation, and during aggregation does matched routing followed by conflict-aware projection. Experiments on PASCAL VOC, MS-COCO, and NUS-WIDE show PathwayFed pulling further ahead as client specialization increases, and the authors add route-permutation, normalization, semantic-mixing, Dirichlet-skew, partial-participation, and pathway-structure analyses to back up the mechanism.

Critical strengths
  1. Beyond the usual benchmark comparisons, the authors add fixed-capacity semantic-mixing controls, route permutation, Dirichlet heterogeneity, normalization, routing-strength interpolation, and partial participation, which makes the empirical case much stronger than a plain performance comparison.

  2. Across all three multi-label benchmarks, PathwayFed is especially strong when client specialization is severe, with big CF1 gains and competitive or better mAP/OF1, and the results are reported over three seeds with ablations that generally back up the routing, sparse refinement, and conflict projection components.

Critical weaknesses
  1. The pathway is defined operationally through unit-to-class activations and weights, and the authors themselves describe several analyses as not establishing semantic independence or an actual entanglement score, so what the evidence supports is more concentrated and client-specific pathway usage, and the stronger "disentanglement" wording may be claiming more than has been shown.

  2. Matched routing largely reweights parameter groups by update magnitude, and the second stage projects conflicting updates, which is conceptually close to existing selective and conflict-aware aggregation ideas, so an apples-to-apples comparison with generic fine-grained weighting plus an existing conflict-removal rule would better show what the pathway formulation uniquely buys.

  3. The evaluation is also strongly tied to a frozen CLIP representation and lightweight prediction heads. Since pathway correspondence assumes a common frozen feature coordinate system, it is unclear whether the mechanism still makes sense when the encoder is jointly trained and client feature spaces drift.

  4. The paper motivates heterogeneous "local label spaces," but the formal setup still assumes a shared global C-class output space. The experiments create heterogeneous label support and co-occurrence patterns, yet every client ultimately uses the same global class coordinates, so the method does not seem to directly handle genuinely different or only partially aligned label vocabularies, and the scope of the motivation should be clarified.

  5. One claimed client-side contribution is also not independently ablated. Dimensional expansion is presented as an important mechanism for refining coarse pathways, but the paper explicitly says it is not isolated in the ablation.

Questions for authors

Please see Critical Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.