- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes SPOS, a client-relatedness measure for federated learning: each client sends only the indices of its top-K (with K=10% typically) layer-normalized update coordinates, and similarity is their overlap. It’s evaluated on relatedness retrieval, clustered personalized FL and checkpoint retrieval, with four model/dataset pairs and 30 clients.
Critical strengths
Estimating client relatedness when secure aggregation hides individual updates is a real problem in FL, and SPOS is a simple primitive that plugs into existing pipelines.
Critical weaknesses
-
SPOS doesn’t beat what it replaces. In clustered FL it matches dense-update cosine within about 1 point, often inside the confidence interval, and on ResNet18 Dir(0.1) retrieval it’s clearly worse (R@6 0.56 vs 0.70). It’s not clear what SPOS brings over cosine besides hiding values, which secure aggregation already does for the aggregation itself.
-
The main intuition behind the proposed method isn’t new. IndexGuard already compares clients through the overlap of their top-K salient update indices, and FedSelect (CVPR 2024) and TIES-Merging (NeurIPS 2023) select parameters by update magnitude. The contribution reduces to using this overlap as a clustering score.
-
Key baselines are missing. PACFL (Vahidian et al., 2023) is cited but not compared, although it clusters clients from a few principal vectors per client and targets heterogeneity beyond label skew, which is exactly SPOS’s claimed goal. Without it, the case for sending tens of MB of indices per client per round (~44 MB for BERT) isn’t made.
-
The evaluation mostly measures label-skew recovery. The oracle is label-histogram divergence on synthetic Dirichlet/pathological splits, where relatedness is by definition label similarity. Feature and concept shift, where SPOS should matter, get one small table. I suggest to highlight it instead.
-
The scale is small: 30 clients, 10 rounds for the similarity experiments, synthetic splits, and CIFAR-10 accuracy around 33% on pathological splits. There’s no natural-partition FL benchmark and no PEFT/LoRA or foundation-model setting, which is where an index-only signal would matter (which is also the current standard).
Questions for authors
N/A