Robust Statistical Learning from Streaming Data: An Adaptive Transfer Perspective

Statistical online learning is challenging when streaming batches may follow different models. This paper studies a contamination setting in which each batch follows a central model with high probability, and a batch-specific model otherwise. To address this online heterogeneity, we approach the problem from a transfer learning perspective. Starting with a point estimation example, we first develop a robust one-pass algorithm that simultaneously updates an estimate of the central parameter, and estimates the parameter of each current batch. We then extend this online method to a linear regression problem and establish time-uniform error bounds: When the current batch follows the central model, the updated central estimator becomes increasingly accurate until reaching a contamination-dependent error floor. When the current batch follows a heterogeneous model, the proposed method adaptively transfers information from previous estimates whenever beneficial, yielding a batch-specific estimator that is potentially more accurate than purely local estimation. The method also provides an online heterogeneity detection procedure. Numerical experiments demonstrate the benefits of our robust online method for learning both central and batch-specific models.