- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies an important potential problem in today's FL personalization: preferential concept shift, in which clients weigh features in a different way, despite the labels.
Critical strengths
- This paper studies an important potential problem in today's FL personalization: preferential concept shift, as far as I know, a missing point in today's existing FL works.
- The illustrations and figures are easy to read and helpful (Figure 1).
- The structure is very clean. The author well defined research questions within the contributions, and the experiment design also well covers those topics. The author also provided comprehensvie supporting materials.
Critical weaknesses
-
I generally like the idea of this paper and recognize its novelty. My major concern is the terminology used throughout the paper. The title refers to “beyond label calibration”, but it is unclear to me what exactly “label calibration” means in the context of FL. The term “calibration” first appears in Line 116, but I could not find a clear definition. In addition, another potentially confusing term is “order”, which usually implies a sequential relationship among results, although I assume this is not the intended meaning in this paper. I suggest that the authors define these terms more precisely or consider using alternative terminology.
-
With that said, personalized federated learning (pFL), rather than vanilla FL, is typically used to address client heterogeneity, including the type of concept drift considered in this paper, i.e., changes in . The related-work discussion appears to omit several representative FL approaches that seem relevant to this setting. To name a few:
- FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering
- Federated Learning with Profile Mapping under Distribution Shifts and Drifts
- Federated Learning with Hierarchical Clustering of Local Updates to Improve Training on Non-IID Data
- Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints
I believe it is necessary to discuss how the proposed method relates to these baselines.
-
Following the second comment, I do not necessarily think that an extensive additional benchmark is required, since the structural assumptions differ substantially. However, I suggest that the authors clarify why IFCA is considered a sufficient representative of cluster-based personalization methods. IFCA may be a canonical baseline, but it is not necessarily representative of the current state of clustered FL under conditional heterogeneity. Would it be possible to include another, more recent baseline?
Questions for authors
Please see Critical Weaknesses.