- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Fairness is a critical factor affecting the performance and efficiency of federated learning (FL). This paper proposes FedICIL, an instance-consistency-based intervention learning framework that aims to address fairness issues at both the intra-client and inter-client levels. The authors evaluate FedICIL across diverse datasets and baseline methods, demonstrating its effectiveness in mitigating unfairness while achieving a favorable trade-off between fairness and predictive performance.
Critical strengths
- Important Research Problem and Novel Perspective. This paper addresses fairness in federated learning, which is an important and enduring research challenge. The authors provide a novel perspective on decomposing the sources of unfairness in FL and introduce instance-level optimization and evaluation as a fundamental approach to mitigating these issues. This instance-centric perspective represents one of the main contributions of the paper.
- Clear Presentation and Comprehensive Experimental Evaluation. The paper is well-organized and logically structured, providing a clear explanation of the underlying causes of unfairness in FL and the motivations behind the proposed framework. Additionally, the figures and experimental results are presented in a visually appealing and informative manner. The evaluation across diverse datasets and baseline methods provides empirical evidence supporting the effectiveness of FedICIL.
Critical weaknesses
-
Unclear Definition of Instance and Potentially Prohibitive Computational Overhead. The paper does not explicitly define what constitutes an “instance,” despite its central role in the proposed framework. Based on the description in Section 4.1, an instance appears to correspond to an individual sample in classification tasks. However, this raises significant concerns regarding computational efficiency. Specifically, if instance consistency must be computed separately for each sample during training or inference, as suggested by Equation (2) in Section 4.1, the resulting computational overhead could be substantial and potentially prohibitive in practical deployments. A detailed computational complexity analysis is therefore necessary. Furthermore, Section 4.1 introduces a neighborhood construction procedure in which each neighborhood contains the top-H most similar instances sharing the same label. It remains unclear whether this procedure requires traversing the entire training dataset to identify these neighbors. If so, the associated computational and memory overhead could further limit the practical applicability of FedICIL. The authors should clarify the implementation details and provide a quantitative analysis of the computational costs associated with instance-consistency estimation and neighborhood construction.
-
Limited Scalability Analysis and Insufficient Hyper-parameter Evaluation. Another concern relates to the scalability of the proposed Sensitivity-Guided Parameter Update mechanism. It is unclear whether this procedure requires estimating the sensitivity of each global model parameter individually. If so, its computational complexity and effectiveness may be strongly influenced by the model architecture and parameter scale. The authors should clarify whether the proposed mechanism remains feasible for large-scale models, particularly large language models (LLMs). Moreover, the proportion of parameters selected by the update mask, controlled by the top-\rho criterion, is a critical hyper-parameter that may substantially influence both model performance and computational efficiency. However, the experimental evaluation lacks a sensitivity analysis of this parameter. Finally, the experiments are conducted using relatively small models and a limited number of clients. These settings are insufficient to demonstrate the scalability and practical effectiveness of FedICIL in large-scale federated learning deployments.
Questions for authors
- Could the authors clarify the precise definition of an “instance” in FedICIL? Does it refer to an individual training sample in classification tasks? If so, does the instance-consistency estimation in Equ. (2) require computations for each sample individually?
- In Section 4.1, each neighborhood is constructed using the top-H most similar instances sharing the same label. Does this process require traversing the entire local training dataset to identify these neighbors? How does the computational cost scale with the number of training samples?
- It seems that the proposed Sensitivity-Guided Parameter Update mechanism require to estimate the sensitivity of each global model parameter individually. If so, how does its computational complexity scale with the model parameter scale, particularly for large language models?
- How does FedICIL perform when deployed with larger models and a substantially greater number of participating clients? Could the authors provide additional scalability experiments or theoretical analysis to demonstrate its applicability to realistic large-scale FL scenarios?