acceptodds
Withdrawn from ICLR 2027

Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Forecasting multiple time series with external covariates requires combining shared predictive structure with series-specific temporal dynamics. We introduce HopFormer (Homogeneity-Pursuit Transformer), a two-stage framework that separates covariate modeling from temporal forecasting. The first stage combines cross-sectional regression experts using Sparsity Pattern Aggregation (SPA) to estimate a covariate-driven component. The second stage adapts a pretrained Transformer through Low-Rank Adaptation (LoRA) to forecast the residuals. For a chronological variant, we establish an end-to-end risk bound under temporal dependence, separating candidate approximation, aggregation error, residual-model input sensitivity, learning and optimization error, and dependence costs. Averaging relative improvements equally across six datasets and ten forecasting baseline models, including those with reported state-of-the-art performance, HopFormer reduces MASE by 7.64% and MAPE by 4.79%. These findings demonstrate the potential of our proposed covariate-driven residual modeling as a modular approach to improving pretrained forecasters, particularly in covariate-informative regimes.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Sparsity Pattern Aggregation, Foundation Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ss1wOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a two stage framework for time series forecasting named HopFormer where the external covariates are available. They separate the covariates prediction from the residual in the forecasting. The stage 1 fits regression experts and recombines them later through sparse pattern aggregation and softmax weights, stage 2 uses a fixed pretrained transformer in stage 1 with the Low-Rank Adaptation. They test their method on 6 selected datasets with three designed synthetic and three real world datasets. They also provided theoretical analysis, ablations, sensitivity analysis and compared with other time series foundation models.

Critical strengths

1 This paper provides a detailed implementation and evaluation protocol. Besides, they report both favorable and unfavorable empirical outcomes.

2 This paper provided the ablation controls with the similar Chronos backbone and verified the different ways for the LoRA adaptation.

Critical weaknesses

1 The model seems applicable to the datasets with the covariates, which limits the application scope of this method. The motivation for decomposing the forecasting problem into two stages should be further clarified. If the goal is not to build a universal forecasting model, it is also unclear why the backbone needs to be frozen instead of finetune the whole parameters.

2 The main components are familiar like LoRA, sparse aggregation, even the decomposition into predictable and residual components. The paper needs stronger justification against the contributions that currently appear at component-level novelty.

3 The empirical evidence lacks comparisons on the standard benchmark in time series forecasting like traffic or widely-used electricity, raising concerns about whether the evaluation is sufficiently representative. The strongest empirical result is concentrated on a particularly favorable synthetic setting.

4 The significant tests are not performed over the random seeds, and only three random seeds are not enough to establish the uncertainty across model training. Considering some performance differences are marginal, for example, the MASE on EPF improves only from 0.528 to 0.524 compared with direct LoRA adaptation, the significant test on training are more favorable.

5 The theoretical analysis does not fully match with the actual model setting. For example, the theoretical analysis uses the squared-loss while the implemented methods use the quartile loss.

6 The ablation study doesn’t test the other pretrained backbone, making it unclear whether the proposed setup is specific for the selected Chronos-Bolt backbone and LoRA configuration?

Questions for authors

1 What is the application scope of this work when informative covariates are unavailable? Besides, the rationale to decompose based on the covariates and residuals is not fully convincing, since the model can predict without covariates and sometimes can do better without the covariates.

2 Why the experiments do not include widely used benchmarks like ETT datasets or weather datasets or commonly used Electricity, traffic datasets, they also have the covariates.

3 Why are significance tests performed over rolling forecast origins rather than over independent training runs? Given that only three random seeds are used and some matched-backbone differences are small, could the authors provide additional runs or statistical analysis over training randomness?

4 How dependent is the proposed framework on the choice of a pretrained backbone?

Reviewer_SrBHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

A shared covariate regressor (SPA over eight experts) is fitted first, and Chronos-Bolt Small is then LoRA-tuned on the residuals. The paper adds a risk bound for a chronological variant and tests on three real and three synthetic panels.

Critical strengths
  1. The evaluation setup is sound: rolling origins, three seeds, block bootstrap with Holm correction.
  2. The right control is provided: the same backbone tuned directly, without Stage I (Table 3).
  3. Negative results and how Table 2 was selected are disclosed.
Critical weaknesses
  1. The method has no new component. The authors say the contribution is "the joint analysis ... rather than a new sparse aggregation or adaptation method" (Sec. 1). Regressing on covariates and then forecasting the residuals is an old recipe, and their own TimesFM XReg baseline does it. The theorem is for a chronological estimator that the experiments never use (they residualize in-sample), so it explains none of the results.

  2. Table 2 was picked after seeing the results (App. D.5: "the main display was narrowed after results had been inspected"). Chronos-2, used frozen, was dropped although it beats HopFormer on EPF (MASE 0.460 vs 0.524), Sales1 (0.320 vs 0.474) and Sales2 (0.214 vs 0.313). The Sales1 and Sales2 columns were dropped too.

  3. The paper's own control shows that residualization does not help. Against direct LoRA on the same backbone, HopFormer loses on Sales1, Sales2, Rossmann and Favorita and ties on EPF. It wins only on Synthetic Electricity, which has oracle covariates and a noise-free target. On Rossmann, Stage I alone (0.686) beats the full model (0.784).

  4. The paper is hard to read: hedges and audit bookkeeping everywhere, two spellings of the method name, and "[cite: 9]" left in Sec. 5.

Questions for authors

See weaknesses.

Reviewer_2yzgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HopFormer, a two-stage framework for incorporating external covariates into pretrained time series forecasting models. The first stage combines regression experts through Sparsity Pattern Aggregation (SPA) to estimate the covariate-driven component of each series. The second stage subtracts this estimate from the observations and adapts Chronos-Bolt with LoRA to forecast the remaining residuals. The final prediction is the sum of the covariate-based prediction and the residual forecast. This design allows the temporal backbone to benefit from external information without directly processing covariate inputs. The paper also analyzes how first-stage estimation errors propagate to downstream forecasting risk under temporal dependence, although the theoretical training procedure differs from the implementation used in the experiments. Evaluations on real and synthetic datasets show benefits in some settings, but the advantage over direct backbone adaptation is highly dataset-dependent.

Critical strengths
  1. The separation between covariate modeling and temporal forecasting is clear and practically useful. It provides a modular way to incorporate external information into a pretrained forecaster without changing its input architecture, while allowing the regression and temporal components to be trained separately.

  2. The evaluation includes covariate-aware forecasting baselines, direct adaptation of the same backbone, and covariate-only predictions. These comparisons provide useful evidence about the roles of the two stages. The paper also reports cases where the proposed decomposition hurts performance, which helps clarify the limits of the approach.

Critical weaknesses
  1. The experiments do not sufficiently separate the benefit of additional covariate information from the benefit of the proposed decomposition and aggregation method. There is no covariate-group ablation, and simpler regressors or aggregation schemes are not evaluated with the same residual backbone and LoRA adaptation. As a result, the additional value of SPA within the complete pipeline remains unclear.

  2. The paper provides limited explanation of when removing the covariate-driven component makes the remaining forecasting problem easier. HopFormer performs worse than direct adaptation of the same backbone on four of the six datasets, while its clearest gain occurs on a synthetic dataset with oracle future covariates and no final independent observation-noise term. The risk analysis does not establish an advantage over direct forecasting, leaving the conditions that justify this decomposition insufficiently understood.

Questions for authors
  1. How much of the observed improvement comes from access to external covariates, and how much comes from the proposed way of combining them with the temporal model?
  • Could the authors provide covariate-group ablations and compare against a simpler regression-plus-residual pipeline using the same covariates, backbone, and LoRA setup? These comparisons would help establish whether the proposed aggregation adds value beyond incorporating the additional information.
  1. When covariate effects differ across series, under what conditions should removing a component estimated by a shared regression model improve temporal forecasting?
  • Could the authors analyze the predictive structure remaining in the residuals and relate it to the differences in covariate effects across series? This would help explain both the successful cases and the datasets where direct backbone adaptation performs better.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.