- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies federated learning for a known target label distribution when only a subset of clients can participate in each round. The objective is to select clients so that training is better aligned with the distribution expected at deployment.FedMLS does not require every round to match the target exactly. Instead, it matches the average label mixture formed by participating clients across rounds. Minimum participation rates limit prolonged exclusion of clients, and virtual queues guide the selections. The paper provides theoretical analyses, an encrypted implementation, and experiments on four datasets, including combinations with FedIR, component ablations, and misspecified target priors.
Critical strengths
-
The method combines target-distribution matching with participation coverage in an implementable framework. It considers both alignment with deployment needs and reliance on a small subset of clients. The contribution lies in this constrained scheduling design, not merely in using KL divergence.
-
Four datasets and eight-run summaries provide substantial empirical evidence. In Table 8, adding matching while retaining minimum participation rates raises accuracy from 33.17% to 36.48% without FedIR, and from 42.92% to 44.61% with FedIR. Matching therefore provides an additional benefit; the gains cannot all be attributed to the participation constraint.
Critical weaknesses
-
The implemented joint algorithm and the explanation of its gains are not yet clearly connected. Equation 1 uses reweighted distributions, whereas Algorithm 2 displays raw distributions. Under Theorem 4's full-support, exact-reweighting assumptions, the label mismatch is already zero. Improved accuracy for the combination is not contradictory, but cannot automatically be explained by further reducing that mismatch. The paper needs to identify the actual selection objective and distinguish matching, participation coverage, and scheduling effects.
-
The privacy claim is broader than the established result. Theorem 2 analyzes the Service's aggregate observations, but Algorithm 2 also gives the Selector client-specific optimization feedback together with its known auxiliary variables. Protection under this different view is not established. The paper needs an argument covering these outputs or a narrower claim; this concern does not establish an actual data breach.
Questions for authors
-
For the FedIR combination, does selection use raw or reweighted distributions? Please specify loss normalization and the treatment of missing classes. Original FedIR uses mini-batch self-normalization; does this implementation do the same? When ideal reweighting has already eliminated label mismatch, what evidence explains the additional selection benefit?
-
On-Demand FL and FedPALS already exploit the target through sampling or aggregation weights. What does the long-run queue add? Existing evidence, or a static-sampling control with the same objective, minimum-rate constraints, and participation budget, could distinguish matching, coverage, and scheduling effects. Cumulative mismatch and participation-coverage trajectories are another option; these are alternatives, not a request for every comparison.
-
How are individual label statistics protected when the Selector knows the auxiliary variables and receives client-specific feedback? Label-aware secure aggregation also treats weights derived from label statistics as confidential. Please explain why the intermediate outputs preserve the claimed protection, or explicitly restrict the guarantee to the Service's aggregate view. This question concerns the stated non-colluding model.
References:
[1] Edvin Listo Zec, Adam Breitholtz, and Fredrik D. Johansson. Overcoming label shift with target-aware federated learning. arXiv:2411.03799v2, revised 2025.
[2] Isu Jeong and Seulki Lee. On-Demand Federated Learning for Arbitrary Target Class Distributions. AISTATS 2024, PMLR 238.
[3] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Federated Visual Classification with Real-World Data Distribution. ECCV 2020.
[4] Aftab Akram, Harry N. H. Pham, Melek Önen, and Clémentine Gritti. SAAFL: Secure Aggregation for Label-Aware Federated Learning. IFIP SEC 2025. DOI: 10.1007/978-3-031-92882-6_3.
We thank the reviewer for the detailed and technically precise questions.
Reply: When applying FedIR, FedMLS uses the reweighted label distribution for the selection. We follow FedIR’s mini-batch normalization of the importance weights. A class that is missing from client n is given zero weight. This has no effect on training since the client has no samples of that class (fn,k=0).
Why Eq. (1) is not zero. If every client held every class, then D=0, as the reviewer notes. However, In our experiments, clients hold on average 5.89 of 1,203 classes (iNaturalist-Users-120k) and 53.70 of 79 classes (Shakespeare). Eq. (1) then becomes D(Sˉ,λ)=−∑kptest,klogcoverage of class k(M1n∑SˉnZn1[k∈Kn]).
where Kn is the support of client n's label distribution and Zn=∑j∈Knptest,j.
This objective is zero only if every test class receives its full target mass from the selected clients. It heavily penalizes classes that are frequent at in test set but held by few selected clients. FedIR corrects the label proportions within each client but cannot supply classes missing from the selected clients. The remaining shift D(Sˉ,λ) can only be reduced by choosing which clients participate, which is what FedMLS does.
Reply: The problem can be formulated as static-probability sampling with the same objective and constraints. However, in the cross-device FL setting, client availability is random and its distribution must be known for this approach. In contrast, drift-plus-penalty framework dynamically adjusts to the client availability with a provable bound on the objective.
Reply: The Selector receives the scores h(r)=MVp⊤u(r) in plaintext over many iterations, and it knows its own a(r). This is sufficient to reconstruct pn. We therefore revise the protocol so that the Selector never sees h or a in plaintext.
Revised protocol (two-party secret sharing). We split a, h, and the queue A into additive shares x=xSel+xSvc, held by the Selector and the Service, respectively. Each share is individually masked by random noise, so neither party ever holds these quantities in plaintext. The protocol changes as follows:
Since selection is model-independent, this computation can run ahead of training. It adds O(N) secure comparisons per inner iteration and one secure top-M selection per round.