Online Multi-Source Transfer Learning under Shifting Source Reliability

Transfer learning leverages labeled source data to improve online prediction when labeled target data are limited. However, both target and source distributions may change over time. A source that helps early in the stream may then become unhelpful, making fixed transfer weights unreliable. We propose Adaptive Multi-Source Transfer Learning (AMTL), which combines a target-only predictor with multiple source-specific transfer predictors. After each target batch, AMTL updates predictor weights based on their losses and uses a structured transition to adjust how much to transfer and which sources to trust. The theoretical analysis establishes a pathwise cumulative-loss bound relative to target-only prediction and examines the sources of error for sparse logistic models. Experiments on synthetic streams, CIFAR-10, and hospital records show that AMTL performs well in online prediction relative to existing methods.