- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies decentralized learning with tabular foundation models(TFMs), where clients cannot share their raw data. The core of the proposed framework is to let different clients make predictions on a shared unlabeled dataset and aggregate these predictions to generate pseudo-labels, which are used to enrich their local contexts. The paper also gives theoretical results on variance reduction and convergence, and evaluates the method on several datasets under IID and non-IID settings.
Critical strengths
- The empirical evaluation is fairly broad, covering both classification and regression tasks, IID and non-IID settings and different communication topologies.
Critical weaknesses
- The main idea of D-ICL is closely related to existing consensus methods in FL [1].
- The baseline comparison is limited. Since D-ICL mainly relies on prediction sharing, more direct comparisons with federated distillation or other prediction-sharing methods are needed to show its advantage[2,3,4,5].
Refs:
[1] FedHarmony: Harmonizing Heterogeneous Label Correlations in Federated Multi-Label Learning
[2] FedMD: Heterogenous Federated Learning via Model Distillation
[3] Ensemble Distillation for Robust Model Fusion in Federated Learning
[4] Federated Model Distillation with Noise-Free Differential Privacy
[5] IMFLKD: an incentive mechanism for decentralized federated learning based on knowledge distillation
Questions for authors
- How is the shared unlabeled query pool obtained in practice?
- How much of the gain comes from iterative context enrichment rather than the initial consensus? D-ICL@0 already benefits from prediction averaging and the added value of repeated pseudo-labeling is not fully clear.
- How does D-ICL compare with simply labeling a small part of the shared pool under a similar budget?