Multi-Mask Online Ensemble Time Series Forecasting

Time series forecasting is increasingly required to operate on non-stationary data streams, where models must adapt online as temporal patterns evolve. Although Transformers have achieved strong performance in offline forecasting by relying on flexible all-to-all attention, such weak inductive bias can be less reliable in online settings. We identify three core mismatches: Over-extended receptive fields that capture spurious correlations. Single static architectures that fail to decouple heterogeneous temporal patterns. Slow adaptation to abrupt concept drift. To address these challenges, we propose MiMe, a Multi-Mask online ensemble framework for time series forecasting. MiMe introduces Pattern-induced Masking to impose diverse temporal inductive biases on self-attention, thereby constraining receptive fields for heterogeneous temporal patterns. It further employs Shared Backbone Training to integrate multiple masked forecasters within a single Transformer backbone, enabling parameter-efficient decoupling of heterogeneous patterns. Finally, online ensembling dynamically reweights masked forecasters through online convex optimization, enabling rapid strategy-level adaptation to concept drift with an $O(\log T)$ regret guarantee. Extensive experiments demonstrate MiMe's strong forecasting performance and robustness against state-of-the-art baselines. Code is available at \https://github.com/anonymous7538/AnonProj.