- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes GloReFair, a globally coordinated sample-reweighting framework for FL, which aims to address global fairness disparities by translating global fairness objectives into sample-level attention signals during local training. By coordinating sample reweighting across clients, GloReFair seeks to achieve fairness correction through decentralized optimization.
Critical strengths
- The paper addresses fairness in federated learning, which is an important and practically relevant research problem. The proposed approach of translating global fairness objectives into sample-level reweighting during local training provides a reasonable and potentially effective mechanism for fairness-aware federated optimization.
- The paper provides theoretical analysis to support the effectiveness of the proposed method under specified assumptions and constraints.
Critical weaknesses
- The theoretical analysis primarily focuses on binary classification tasks, leaving the effectiveness and generalizability of GloReFair to more complex learning scenarios insufficiently established. In particular, under limited local data availability and highly non-IID distributions, estimating group-specific rates in multi-class classification tasks may suffer from substantial statistical variance. The authors do not adequately demonstrate whether the proposed fairness correction mechanism remains reliable under such challenging conditions.
- The residual signals are computed using the arithmetic mean of group-specific rates, which may overlook inherent differences in prediction difficulty across tasks or classes. Additionally, assigning equal importance to different groups without considering their respective sample sizes may introduce estimation bias or amplify the influence of statistically unreliable groups.
- The proposed sample-reweighting mechanism appears to require sample-level attention computation during local training, potentially introducing substantial computational overhead. In particular, if the attention weights must be calculated individually for each training sample in every communication round, the additional computation could significantly increase local training costs and reduce the overall efficiency of federated optimization.
Questions for authors
- The theoretical analysis primarily considers binary classification tasks. How does the proposed framework generalize to multi-class classification problems, particularly under highly non-IID data distributions and limited local sample sizes?
- The residual signals are calculated using the arithmetic mean of group-specific rates. What is the theoretical or empirical justification for assigning equal importance to different groups or tasks, particularly when they exhibit different levels of prediction difficulty and substantially different sample sizes?
- How does GloReFair ensure reliable fairness estimation when certain sensitive groups or classes contain very few samples on individual clients?
- Does the proposed attention-based reweighting mechanism require computing an individual attention weight for every training sample in each communication round? If so, what is the additional computational complexity compared with standard local training? Could the authors provide a detailed analysis of the per-round computational overhead, including additional training time and memory consumption?
6 comments
[W2, Q2] Distinction between group weighting and statistical reliability
Thank you for raising the distinction between group weighting and statistical reliability. We use the arithmetic mean because it provides a symmetric reference for comparing the conditional prediction rates of different groups. As defined in Section~3, TPRs are averaged across groups within the positive-label slice, and FPRs are averaged separately within the negative-label slice. Subtracting this reference produces centered residuals that preserve the differences between group rates and sum to zero within each slice. All residuals vanish exactly when the corresponding group rates coincide, directly matching the EO/EOD parity objective. The construction compares groups under the same label condition, without requiring positive and negative classes to have the same prediction difficulty.
Equal importance reflects the equal standing of sensitive groups under the adopted fairness criteria. EO/EOD requires the corresponding conditional rates to agree across groups, regardless of their prevalence in the dataset. A smaller group has the same claim to fair treatment as a larger group. We therefore give each active group equal weight in defining the fairness reference. Group size determines how much evidence is available to estimate a rate; it does not determine the importance of that group's fairness. Differences in prediction difficulty can affect the attainable fairness--accuracy trade-off, while the parity target remains the same.
Sampling uncertainty is a separate statistical issue. When a group-label channel has few observations in the pooled dataset, its empirical rate can be an imprecise estimate of the population rate. This limitation is shared by finite-sample estimation of group fairness, including centralized EO/EOD evaluation. Its existence does not by itself establish estimation bias caused by equal-group weighting. GloReFair uses sample counts explicitly: each channel rate is computed from prediction statistics and counts aggregated across all clients. For a fixed common model and prediction rule under full participation, Supplement~B establishes that these rates equal those computed centrally on the pooled data. Consequently, client partitioning introduces no additional error into the empirical rates; the remaining sampling uncertainty depends on the global support available in each channel.
[W3, Q4] Computational overhead of attention computation
Thank you for raising the concerns on computational overhead of GloReFair. GloReFair computes each sample's weight once per reweighting round and reuses it throughout all local SGD steps. Clients evaluate the sample losses at the synchronized model and apply the closed-form weighting rule element-wise, with gradients disabled during weight preparation. For a fixed model, weight construction and storage scale linearly with the local sample count. Each stored weight occupies one FP32 value.
We measured the computational cost on Adult with LR classifier, five fully participating clients, joint Dirichlet heterogeneity H = 0.2, and the EOD objective. The three methods shared data partitions, initial models, and batch schedules. The following table reports the aggregate computation time of the five-client simulation, including weight preparation, local optimization, aggregation, fairness feedback, and controller updates.
Table: Per-round computation time (seconds) across local training budgets.
Entries are means ± sample standard deviations over three seeds.
| Local epochs | FedAvg | FedFB | GloReFair |
|---|---|---|---|
| 1 | 0.079 ± 0.001 | 0.100 ± 0.003 | 0.144 ± 0.0014 |
| 5 | 0.393 ± 0.003 | 0.446 ± 0.002 | 0.599 ± 0.004 |
| 10 | 0.769 ± 0.010 | 0.872 ± 0.011 | 1.180 ± 0.010 |
| 20 | 1.532 ± 0.003 | 1.719 ± 0.018 | 2.319 ± 0.014 |
The results identify where this computation is spent. At 10 local epochs, weight preparation, including its forward evaluation, takes 10.900 milliseconds per round and accounts for 0.92% of GloReFair's total algorithm time. Most of the additional time occurs in local optimization, which includes weighted-loss evaluation, and the proximal penalty. The measured overall overhead is 53.4% relative to FedAvg and 35.4% relative to FedFB. Thus, the principal computational cost lies in the local optimization stage, while the sample weights are constructed once and reused across the round.
Table: Cost breakdown at 10 local epochs.
Stage times are means over three seeds in milliseconds per round. Peak memory is reported as mean ± sample standard deviation in MiB and covers the entire process.
| Component | FedAvg | FedFB | GloReFair |
|---|---|---|---|
| Weight preparation (ms) | 0.000 | 0.000 | 10.900 |
| Local optimization (ms) | 768.781 | 859.930 | 1152.877 |
| Aggregation and distribution (ms) | 0.520 | 0.520 | 0.525 |
| Fairness feedback (ms) | 0.000 | 10.914 | 15.227 |
| Server control (ms) | 0.000 | 0.147 | 0.705 |
| Peak process memory (MiB) | 569.3 ± 0.1 | 570.1 ± 0.5 | 573.0 ± 0.5 |
[W1, Q1, Q3] Concerns on binary classification and limited local data
Thank you for your comments. Our binary classification setting follows the established definitions of equal opportunity and equalized odds, providing a precise setting for studying global group fairness under federated heterogeneity. A concrete route to multiclass learning is a classwise one-versus-rest formulation: each binary subtask would have its own group-label channels, channel prices, and pooled fairness residuals. This extends the same principle of translating global fairness feedback into local training signals.
GloReFair directly addresses sparse local group-label coverage through pooled statistics. Clients contribute sample counts, positive-prediction counts, and score sums, from which the server computes global rates. A locally absent channel contributes zero statistics and requires no local rate estimate. Supplement B establishes that, for a fixed pooled dataset, common model, prediction rule, and mixture coefficient under full participation, the fairness feedback is invariant to client partitioning. Consequently, dividing the same data into smaller local subsets leaves this feedback unchanged. Its statistical support is determined by the pooled sample count in each channel.
The experiments provide complementary evidence at the training level. Supplement H.4 examines increasing the number of clients from 5 to 25 while keeping the total training set fixed, directly testing greater local data fragmentation. The following table presents the original EOD results under the strongest evaluated joint Dirichlet heterogeneity, where both demographic composition and label distributions vary substantially across clients.
Table: EOD results under joint Dirichlet heterogeneity with H = 0.2.
A dash denotes failed runs.
These results demonstrate a stable fairness-accuracy trade-off under severe client heterogeneity, consistent with the partition-invariant construction of its global fairness feedback.