Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

Forecasting multiple time series with external covariates requires combining shared predictive structure with series-specific temporal dynamics. We introduce HopFormer (Homogeneity-Pursuit Transformer), a two-stage framework that separates covariate modeling from temporal forecasting. The first stage combines cross-sectional regression experts using Sparsity Pattern Aggregation (SPA) to estimate a covariate-driven component. The second stage adapts a pretrained Transformer through Low-Rank Adaptation (LoRA) to forecast the residuals. For a chronological variant, we establish an end-to-end risk bound under temporal dependence, separating candidate approximation, aggregation error, residual-model input sensitivity, learning and optimization error, and dependence costs. Averaging relative improvements equally across six datasets and ten forecasting baseline models, including those with reported state-of-the-art performance, HopFormer reduces MASE by 7.64% and MAPE by 4.79%. These findings demonstrate the potential of our proposed covariate-driven residual modeling as a modular approach to improving pretrained forecasters, particularly in covariate-informative regimes.