acceptodds
Withdrawn from ICLR 2027

Delving into Instances: A Unified Perspective for Fair Federated Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Fairness in federated learning (FL) is commonly studied at the client or group level to reduce performance disparities across clients or groups. Such coarse-grained views overlook an important source of unfairness: *similar instances may receive inconsistent predictions, even within the same client*. In this work, we find that *instances provide a unified and finer-grained perspective for fairness in FL*. Instance-level prediction consistency not only characterizes *intra-client unfairness* among similar instances within a client, but also provides a basis for identifying and mitigating *inter-client unfairness* through differences in the consistency of instances across clients. Based on this insight, we propose Federated Instance-Consistency Intervention Learning (FedICIL), a unified instance-level framework for fair FL. Within each client, FedICIL constructs intra-client neighborhoods according to instance similarity and explicitly regularizes predictions to be consistent within each neighborhood. To make the intervention more targeted under limited local update budgets, FedICIL further identifies parameters that are sensitive to instance-level prediction inconsistency and selectively updates them. Beyond the client level, clients communicate a lightweight instance-consistency signal, which is used to adapt aggregation weights according to the remaining inconsistency of their local predictions. In this way, the same instance-level signal is exploited to address both intra-client inconsistency and inter-client performance disparities. Experiments on diverse benchmarks demonstrate that FedICIL achieves a better trade-off among predictive performance, inter-client fairness, intra-client fairness, and group fairness compared with existing methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Fair Federated Learning, Instance-Level Fairness, Prediction Consistency

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_qqc9OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Fairness is a critical factor affecting the performance and efficiency of federated learning (FL). This paper proposes FedICIL, an instance-consistency-based intervention learning framework that aims to address fairness issues at both the intra-client and inter-client levels. The authors evaluate FedICIL across diverse datasets and baseline methods, demonstrating its effectiveness in mitigating unfairness while achieving a favorable trade-off between fairness and predictive performance.

Critical strengths
  1. Important Research Problem and Novel Perspective. This paper addresses fairness in federated learning, which is an important and enduring research challenge. The authors provide a novel perspective on decomposing the sources of unfairness in FL and introduce instance-level optimization and evaluation as a fundamental approach to mitigating these issues. This instance-centric perspective represents one of the main contributions of the paper.
  2. Clear Presentation and Comprehensive Experimental Evaluation. The paper is well-organized and logically structured, providing a clear explanation of the underlying causes of unfairness in FL and the motivations behind the proposed framework. Additionally, the figures and experimental results are presented in a visually appealing and informative manner. The evaluation across diverse datasets and baseline methods provides empirical evidence supporting the effectiveness of FedICIL.
Critical weaknesses
  1. Unclear Definition of Instance and Potentially Prohibitive Computational Overhead. The paper does not explicitly define what constitutes an “instance,” despite its central role in the proposed framework. Based on the description in Section 4.1, an instance appears to correspond to an individual sample in classification tasks. However, this raises significant concerns regarding computational efficiency. Specifically, if instance consistency must be computed separately for each sample during training or inference, as suggested by Equation (2) in Section 4.1, the resulting computational overhead could be substantial and potentially prohibitive in practical deployments. A detailed computational complexity analysis is therefore necessary. Furthermore, Section 4.1 introduces a neighborhood construction procedure in which each neighborhood contains the top-H most similar instances sharing the same label. It remains unclear whether this procedure requires traversing the entire training dataset to identify these neighbors. If so, the associated computational and memory overhead could further limit the practical applicability of FedICIL. The authors should clarify the implementation details and provide a quantitative analysis of the computational costs associated with instance-consistency estimation and neighborhood construction.

  2. Limited Scalability Analysis and Insufficient Hyper-parameter Evaluation. Another concern relates to the scalability of the proposed Sensitivity-Guided Parameter Update mechanism. It is unclear whether this procedure requires estimating the sensitivity of each global model parameter individually. If so, its computational complexity and effectiveness may be strongly influenced by the model architecture and parameter scale. The authors should clarify whether the proposed mechanism remains feasible for large-scale models, particularly large language models (LLMs). Moreover, the proportion of parameters selected by the update mask, controlled by the top-\rho criterion, is a critical hyper-parameter that may substantially influence both model performance and computational efficiency. However, the experimental evaluation lacks a sensitivity analysis of this parameter. Finally, the experiments are conducted using relatively small models and a limited number of clients. These settings are insufficient to demonstrate the scalability and practical effectiveness of FedICIL in large-scale federated learning deployments.

Questions for authors
  1. Could the authors clarify the precise definition of an “instance” in FedICIL? Does it refer to an individual training sample in classification tasks? If so, does the instance-consistency estimation in Equ. (2) require computations for each sample individually?
  2. In Section 4.1, each neighborhood is constructed using the top-H most similar instances sharing the same label. Does this process require traversing the entire local training dataset to identify these neighbors? How does the computational cost scale with the number of training samples?
  3. It seems that the proposed Sensitivity-Guided Parameter Update mechanism require to estimate the sensitivity of each global model parameter individually. If so, how does its computational complexity scale with the model parameter scale, particularly for large language models?
  4. How does FedICIL perform when deployed with larger models and a substantially greater number of participating clients? Could the authors provide additional scalability experiments or theoretical analysis to demonstrate its applicability to realistic large-scale FL scenarios?
Reviewer_rE4UOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces FedICIL, a framework for achieving individual fairness within clients in a federated learning setting. The authors point out that the existing literature on fairness mainly concerns achieving fairness at a group or client level, i.e., ensuring that overall utility/fairness across clients is preserved. This does not necessarily mean that within a client, similar individuals achieve similar outcomes. To this end, the authors propose to construct intra-client neighborhoods based on instance similarity and explicitly regularize predictions to be consistent within each neighborhood. The framework is deployed across datasets/tasks spanning multiple modalities, where it is demonstrated to achieve better performance compared to existing baseline methods.

Critical strengths

The paper is easy to follow and investigates an interesting problem in the context of fairness in federated learning.

Proposes a novel regularization method to achieve intra-client performance homogenity.

Evaluation is performed across datasets and tasks spanning multiple modalities.

Critical weaknesses

The evaluation part is critically weak. CoLA, QNLI, RTE datasets have nothing to do with fairness. In fairness research, a critical component of the datasets is the presence of a sensitive attribute. For example, CoLA measures grammatical acceptability of a sentence. What has this to do with fairness? The same goes for MNLI, MRPC. These are all standard NLP benchmarks and are not related to fairness. The authors should pick up datasets specifically designed to investigate bias such as Stereoset (https://huggingface.co/datasets/McGill-NLP/stereoset) or Biosbias (https://huggingface.co/datasets/LabHC/bias_in_bios) inter alia.

While the paper motivates with the heterogenous data across clients, it is unclear if their data distribution across clients is actually so. In lines 299-300, the authors mention data being distributed across five heterogenous clients. How do I know if the clients are indeed heterogenous? More importantly, how did you split the data across clients so as to ensure the data distribution is indeed heterogenous? Similarly, why exactly 5 clients in the case of NLU experiments and 20 for the vision datasets? Was the data split equally across the clients? Ideally, one needs to repeat these experiments with different client configurations to meaningfully demonstrate the effectiveness of the approach.

None of the results provide any statistical tests to demonstrate if the achieved improvements are indeed significant. Also, in Table 1, why do you show improvements over the FedAvg algorithm, which was trained in a standard way devoid of any specific regularization to achieve individual fairness? It is misleading and inflates the achieved improvements. It should be with respect to the most competitive baseline.

There are also no theoretical justifications for the choice of the regularization scheme introduced or its convergence guarantees.

Questions for authors

What is the computational complexity of the proposed framework? What is the per-round computation time compared to the baseline methods?

How do you measure the heterogeneity of the data distribution across clients?

Do you have any intuitions regarding the convergence guarantees of the proposed framework?

Reviewer_YvU9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FedICIL, an instance-level framework for fair federated learning that combines neighborhood-based prediction consistency, sensitivity-guided parameter updates, and consistency-based client aggregation. Experiments on language understanding, cross-domain image classification, and medical imaging demonstrate favorable overall trade-offs among predictive performance, inter-client fairness, intra-client consistency, and demographic fairness.

Critical strengths
  • The paper addresses a meaningful limitation of aggregate fairness evaluation. Client-level performance statistics can overlook differences in how similar instances are treated within each client. Bringing instance consistency into the evaluation alongside client disparities and demographic gaps is therefore valuable.
  • The framework integrates local intervention and global aggregation in a coherent way. The same locally computed consistency objective is reused for prediction regularization, sensitivity-based parameter selection, and client aggregation.
  • The empirical evaluation is broad and contains meaningful positive evidence. The experiments span language understanding, cross-domain image classification, and medical imaging, with multiple model architectures and several relevant baseline families.
Critical weaknesses
  • The proposed consistency surrogate is insufficiently justified as a measure of individual fairness. The paper does not establish why the model-dependent geometry captures fairness-relevant similarity, or how anchoring predictions to it avoids preserving existing model biases. Moreover, matching target-class confidence does not control the full predictive distribution in multiclass problems. This leaves an important gap between the implemented surrogate and the individual-fairness interpretation underlying the paper’s central contribution.
  • Limited theoretical guarantees for convergence and individual consistency. FedICIL combines model-dependent neighborhoods, sensitivity-guided parameter masks, and adaptive aggregation weights, but provides no convergence analysis for the resulting coupled training procedure. Moreover, minimizing the target-class consistency loss does not establish the pairwise prediction-consistency condition, particularly for the final aggregated model. Consequently, the paper leaves unresolved under what conditions training converges and whether the resulting model satisfies a quantifiable individual-consistency guarantee.
  • The added value of the neighborhood-based consistency design is not sufficiently isolated. The ablations show that removing the consistency loss reduces performance, but do not establish whether the proposed neighborhood consensus offers benefits beyond simply anchoring each instance’s prediction to its previous global-model prediction. A comparison under matched parameter-update budgets and otherwise comparable training settings would help clarify the specific contribution of the neighborhood-based intervention. Moreover, I recommend adding a baseline that applies an established individual-fairness training method locally and then uses standard FedAvg aggregation
Questions for authors
  1. What justifies using the previous global model’s predictive outputs to define fairness-relevant similarity, particularly when that model may already contain biases? Do the individual-fairness gains persist under an independently motivated similarity metric, and how does the target-class-only objective address full-distribution consistency in multiclass settings?

  2. Under what assumptions does FedICIL’s coupled training procedure converge despite its changing neighborhoods, parameter masks, and aggregation weights? Can the authors establish a quantitative relationship between the optimized consistency loss and the pairwise prediction-consistency condition in Section 3 for the final aggregated model?

  3. How does FedICIL compare with a simpler baseline that anchors each instance’s prediction to its own prediction under the previous global model, using matched parameter-update budgets and otherwise comparable training settings? In particular, does introducing the neighborhood consensus provide additional fairness benefits beyond those obtained by stabilizing predictions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.