- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Online Asynchronous Federated Conformal Prediction (OAFCP), a framework for uncertainty quantification in federated learning with streaming data and heterogeneous client computation speeds. The method jointly updates predictive models and conformal thresholds locally, while asynchronously aggregating client updates at fixed server intervals using participation-frequency-aware weights. The authors establish theoretical results concerning long-run coverage and asymptotic calibration under certain assumptions. Experiments on synthetic data and six real-world datasets suggest that OAFCP can achieve competitive coverage and prediction efficiency compared with a synchronous baseline, particularly under distribution shifts.
Critical strengths
- The paper addresses the intersection of online conformal prediction and asynchronous federated learning, which is an important yet relatively underexplored setting.
- OAFCP combines online model updates with adaptive conformal calibration and fixed-interval asynchronous aggregation. The client-server communication protocol is clearly motivated, and the proposed framework does not require exchanging raw observations.
- The paper goes beyond proposing an algorithm by investigating both long-run empirical coverage and instantaneous conditional coverage.
Critical weaknesses
- The learning-rate definitions used in the algorithm and theoretical analysis seems to be inconsistent. Section 3.2 specifies local calibration updates with learning rates indexed by each client's observation count. However, Assumption 3 introduces a task-level learning-rate convention based on the initialization round. Furthermore, Theorem 1 defines a reference learning rate involving the server-round index, aggregation interval, and local task size.
- The experimental evaluation compares OAFCP only against the online synchronous federated conformal prediction baseline (OSFCP). It would be important to compare against additional approaches, such as asynchronous federated learning combined with local online conformal calibration, and alternative aggregation-weighting strategies.
- The paper lacks ablation studies examining the individual contributions of asynchronous aggregation, client-frequency compensation, and joint model-threshold updates.
Questions for authors
Can the authors provide stronger baselines and ablation studies? How does OAFCP compare with an asynchronous federated learning method equipped with independent local online conformal calibration? What happens when the participation-frequency correction is removed or replaced with uniform aggregation weights?