- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this work, the authors propose FalOTS, a federated active learning framework that employs orthogonal, target-aware Sharpness-Aware Minimization (SAM) for optimization. During each sample acquisition phase, FalOTS constructs class weights based on the ratio between target priors and the distribution of labels already acquired globally. It computes target-aware gradients for each labeled mini-batch and incorporates the normalized orthogonal residual of these gradients into the SAM perturbation. By integrating global class information into local optimization, FalOTS achieves its objectives without requiring any modifications to the existing sample acquisition rules or server-side aggregation algorithms used in federated active learning. Extensive experiments are conducted to evaluate the performance of the proposed mechanism.
Critical strengths
-
This paper distinguishes between heterogeneity among clients and class imbalance in the aggregate data across all clients, demonstrating that the latter can still persist. This helps clarify the distinction between existing federated SAM methods and the specific problem addressed in this paper.
-
The presentation of the proposed method is clear and the paper is overall well-written.
-
The evaluation of the proposed method is solid and comprehensive.
Critical weaknesses
-
Global prior information may become completely ineffective after local projection. The entire FalOTS framework could thus degenerate into FedSAM.
-
The method discards all parallel components, even when the target gradient points in the opposite direction to the empirical gradient.
-
In Table 1, the Target entry for Focal-SAM is marked with a ×. However, the original Focal-SAM paper explicitly establishes a framework characterized by an imbalanced training distribution, a uniform target distribution during testing, and consistent class-conditional distributions, with the objective of mitigating the risks associated with a uniform test distribution. Therefore, this "×" at most indicates that Focal-SAM does not accept an arbitrary, user-specified as an explicitly configurable input; it cannot be interpreted to mean that the method fails to account for the discrepancy between the training prior and the target prior.
-
In Table 2, multiple medical baselines (Random, KAFAL, LoGo) are evaluated under two heterogeneity settings: and .
-
The absence of category-specific results makes it impossible to directly demonstrate that the gains stem from addressing class imbalance.
Questions for authors
See in weaknesses.
9 comments
Official Author Responses to Reviewer EYHx [2/7]
W1:Global prior information may become completely ineffective after local projection. The entire FalOTS framework could thus degenerate into FedSAM.
This is a legitimate concern, and it can be answered precisely.
When can the correction vanish? On a client's mini-batch, and are two positive combinations of the same per-class mean gradients (: batch label frequencies). They are collinear, so that , only if (a) is constant over the classes present in the batch, i.e., the global labeled prior already matches the target on those classes, in which case reducing to FedSAM is the intended behavior (Prop. 3(2)); or (b) the per-class gradients are themselves collinear, a measure-zero event in . The projection removes a single direction,, out of , and the global information, which enters through , lives in the remaining dimensions. The weights are global by design: with a common , (Prop. 1), so even a client that holds only a few classes up-weights exactly the globally rare classes among them.
Measured during training. We recorded the residual on every client's mini-batches along full FAL runs (, ; 7,920 probes per dataset; Tab. 17 and Fig. 14c of the revised PDF), which is exactly the local projection the reviewer asks about:
Tab. R1: Relative size of the orthogonal residual over all probed local steps. is the sine of the angle between and ; a value of 0 means the step reduces to FedSAM. The zero-guard of Alg. 1 sets when .
| median | 10th pct. | min. | zero-guard triggered | |
|---|---|---|---|---|
| CIFAR-10 | 0.56 | 0.12 | 0.011 | 0% |
| CIFAR-100 | 0.53 | 0.40 | 0.248 | 0% |
The orthogonal part carries about half of the target-weighted gradient's norm in a typical step and is stable across all nine phases (median 0.51–0.61).
Accuracy. Consistently, FalOTS matches or improves over FedSAM in every setting of Tabs. 5–7 and 12 ( from 0.1 to 100); the gain is largest at and smaller at (Fig. 6b), which is what the analysis predicts as the priors approach each other. We added this argument to Sec. 3.3 and the measurements to Appx. J.3.
Official Author Responses to Reviewer EYHx [3/7]
W2:The method discards all parallel components, even when the target gradient points in the opposite direction to the empirical gradient.
Why the parallel component is discarded. The budget along is SAM's own ascent step, which is what makes the method sharpness-aware. Keeping changes SAM's linearized increase (Prop. 3(1)): a positive component only rescales , which Fig. 6c shows is not where the gain comes from, and a negative component would partly cancel the ascent and remove sharpness control exactly when head- and tail-class gradients disagree. The sign is not lost: it enters , and under full anti-alignment the residual vanishes and FalOTS conservatively falls back to SAM. If the target gradient opposed the training gradient as a whole, we believe the remedy would belong in the descent objective rather than in the neighborhood probe.
Does the opposing case occur in practice? We measured it on all clients, rounds and phases (Appx. J.3): occurred in 0 of 15,840 probes on CIFAR-10 and CIFAR-100 as presented in Tab. R2; the median is 0.83–0.85. This is expected, since both are positive combinations of the same per-class gradients. Hence a "keep only when negative" variant coincides with FalOTS at every probed step, while "always keep" (Fig. 6a, "no ortho") is consistently worse. This discussion is now in Sec. 3.3.
Tab. R2: Alignment of the target-weighted and empirical gradients along one full FAL run per dataset (, ; 7,920 probed local steps per run; Appx. J.3 of the revised PDF).
| CIFAR-10 | CIFAR-100 | |
|---|---|---|
| , FalOTS iterates | 0 of 7,920 | 0 of 7,920 |
| , FedSAM iterates | 0 of 7,920 | 0 of 7,920 |
| median (range over the 9 phases) | 0.83 (0.80–0.85) | 0.85 (0.79–0.86) |
Official Author Responses to Reviewer EYHx [1/7]
We thank Reviewer EYHx for the careful review and for appreciating the distinction between client heterogeneity and global class imbalance, the clarity of the presentation, and the solidity of the evaluation. Each point is addressed below. We conducted requested experiments/discussion on (1) Potentially FalOTS degenerates into FedSAM, (2) Concern about discarding all parallel components, (3) Clarification "Target" entry for Focal-SAM, (4) Duplication number and (5) Category-specific results. The corresponding revisions are marked in blue in the revised PDF, Tab. R-numbers refer to the new results (Figures are added directly in the revision) in Appx. J.