acceptodds
Under review as a conference paper at ICLR 2027

CONFIGURING BEFORE PREDICTION: RETHINKING TIME-SERIES FORECASTING TASK FORMULATION

Abstract

Time-series forecasting research has devoted substantial effort to improving how models predict, while largely taking for granted what historical input they should predict from. Yet the historical input is itself a key part of the forecasting task formulation: it determines both what temporal structure is exposed to the predictor and how far into the past that structure extends. In common forecasting practice, these aspects are typically predetermined rather than adapted to the time series and forecast horizon at hand. We therefore propose a framework that jointly configures the temporal structure and temporal extent of the historical input for each channel and forecast horizon before prediction, selecting among reversible temporal representations and candidate historical extents. To make heterogeneous input formulations comparable without repeatedly fitting downstream models, we establish a unified risk characterization of their expected multi-step forecasting behavior on the original data scale, enabling principled configuration directly from the training series. Experiments on public benchmarks show that the resulting configurations consistently improve forecasting accuracy over commonly used fixed configurations across diverse forecasting architectures, including DLinear, PatchTST, and Koopman-based models. The improvements also persist across different choices of linear operators, demonstrating that the benefits are not tied to a particular predictor. Notably, even a simple scalar autoregressive forecaster becomes competitive with substantially more complex forecasting models when equipped with the configured historical input. These results highlight historical input formulation as an important yet underexplored dimension of time-series forecasting and show that configuring what a model predicts from can be as consequential as improving how it predicts.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, training-free, Koopman operator, long-horizon forecasting, autoregressive models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9AtNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper argues that historical input formulation is an important yet underexplored part of time-series forecasting. It considers two dimensions of the input: temporal structure and temporal extent. It proposes a training-data-based procedure for selecting those configurations using a unified criterion for expected multi-step forecasting risk. Experiments on public benchmarks show that the configured inputs improve a scalar autoregressive forecaster over commonly used fixed-input configurations and that configuring the temporal extent can also benefit several other state-of-the-art forecasting baselines.

Critical strengths
  • The paper highlights an important but under-explored aspect of time-series forecasting: how historical inputs are formulated. It treats temporal structure and lookback length as task-dependent choices, rather than fixed conventions. Controlled comparisons show that configuring these inputs improves the performance of a fixed scalar autoregressive forecaster. In a separate benchmark comparison, this forecaster is competitive with several recent forecasting baselines. Together, these results motivate further attention to input formulation as a complement to predictor design.

  • The paper presents a well-structured and intuitive training-only criterion for selecting historical-input configurations. It combines reconstructed multi-step forecasting error estimated using a scalar linear autoregressive reference forecaster, coefficient-estimation uncertainty, and persistent level variation, without requiring repeated fitting of downstream forecasters. The extensive ablations examine the effects of configuring temporal structure and extent, comparing candidates on different scales, and including individual criterion components. These experiments help clarify the role of each design choice and make the evaluation easier to interpret.

Critical weaknesses
  1. The transferability of the full input configuration remains uncertain. The criterion selects structure and extent using a channel-wise scalar autoregressive reference, which may rank temporal structures differently from downstream forecasters. In Table 3, transferring the full (m,p)(m,p) configuration substantially worsens DLinear and PatchTST on Weather dataset, even though transferring pp alone helps both models. The shared-input models also replace channel-wise choices with a majority structure and median extent, making it unclear whether the failure comes from the AR-based criterion or this aggregation rule. Transfer is evaluated on only three datasets and does not include diverse forecasting baselines, such as models explicitly modeling cross-channel dependencies. This limits the evidence for the paper’s broader applicability to multivariate forecasting.

  2. Channel-wise selection leaves the role of cross-channel dependencies unresolved. The criterion evaluates each channel separately and does not use predictive relationships between channels when selecting its input configuration. The experiments do not establish how well this procedure works when cross-channel relationships are important to prediction, which may limit its applicability to multivariate forecasting settings.

  3. The method relies on a heuristic configuration space, and the added value of jointly selecting the two input dimensions is not fully established. It searches over four conventional temporal representations and a pre-specified lookback grid. The results show that adapting both dimensions is useful, but it is less clear whether they need to be selected as a coupled pair. It does not compare against selecting mm and pp independently and then combining them.

  4. The selection criterion may depend strongly on the capacity and inductive bias of its scalar AR reference. Evaluated values, such as the reconstructed multi-step error (VV) are estimated using a channel-wise autoregressive reference. Configurations favored by this reference need not be the best inputs for other forecasting models. Without an alternative reference or a more direct analysis of downstream candidate rankings, it is difficult to assess the reliability of the selection criterion.

Questions for authors

Q1. Could you compare joint selection with independently selecting mm and pp and then combining the two choices? For example, mm could be selected with fixed p=Hp=H, while pp is selected with m=Idm=Id, using the same training-only criterion.

Q2. How sensitive are the selected configurations to the scalar AR reference? Could you evaluate an alternative training-only reference, such as a lightweight rule-based or other forecasting model, or report how the AR-based candidate rankings align with candidate performance under downstream forecasters?

Q3. Could you expand the full (m,p)(m,p) transfer experiment in Table 3 to additional datasets and another recent forecasting architecture?

Q4. Could you analyze whether the method’s gains vary with the importance of cross-channel predictive relationships? A comparison across datasets with weaker and stronger cross-channel dependence would help assess the applicability of channel-wise selection.

7 comments

Response to W1: Full historical input formulation transfer and channel heterogeneity

Thank you for highlighting the uncertainty in full-formulation transfer and the confounding between the reference criterion and the aggregation rule. To investigate this issue, we conducted additional experiments across all four Weather horizons, grouping channels by their training-derived temporal structures and fitting a separate forecaster for each group. This substantially mitigates the degradation for both DLinear and PatchTST.

In the original transfer implementation, channel-specific historical input formulations are reduced to a majority temporal-structure family and a median lookback, which are then shared across channels. This can override channel-specific temporal structures. For example, at longer Weather horizons, precipitation channels originally associated with the identity representation are instead subjected to seasonal differencing. Such reassignment may be unfavorable because inverse differencing can accumulate forecast bias and correlated errors.

To examine whether preserving channel heterogeneity can mitigate the observed degradation, we conducted grouped experiments across all four Weather horizons, H∈96,192,336,720H\in\\{96,192,336,720\\}. At each horizon, channels were grouped according to their training-derived temporal structures mcm_c. Each group received a separately trained forecaster with its corresponding preprocessing. All groups used a common lookback of p=1776p=1776. We retained the original data splits, training recipe, and original-scale evaluation metrics.

Table 1. Four-horizon mean test MSE for shared and grouped historical input formulations on Weather.

Input formulationModels per taskDLinearPatchTST
Fixed L=96L=9610.26700.2579
Fixed L=HL=H10.25000.2333
Configured extent pp only, identity input10.23500.2208
Globally shared full formulation (m,p)(m,p)10.65630.6729
New: grouped by mcm_c, with corresponding preprocessing and p=1776p=177640.22860.2722

The grouped alternative reduces mean MSE from 0.6563 to 0.2286 for DLinear and from 0.6729 to 0.2722 for PatchTST. These results are consistent with preserving channel heterogeneity being a useful mitigation for the degradation observed under global sharing. However, the grouped procedure also changes temporal extent, parameter sharing, and total model capacity. Its additional cost includes training and maintaining four separate sub-models per task.

The remaining differences across predictors are also informative. Grouped DLinear achieves slightly lower mean MSE than its extent-only identity baseline, whereas grouped PatchTST remains worse than both that baseline and the fixed-input baselines. Thus, grouping substantially mitigates the original degradation but does not establish uniformly beneficial full-formulation transfer.

We agree that the reference criterion may rank temporal structures differently from downstream neural forecasters. At the same time, the remaining PatchTST gap cannot be uniquely attributed to that criterion, because the grouping granularity, shared temporal extent, model capacity, and optimization also affect downstream performance. The present experiment does not fully disentangle these factors.

We will include these results and their additional cost, distinguish channel-specific historical input formulation from its adaptation to shared-input architectures, and qualify our claims about full-formulation transfer. We address the reviewer’s complementary concern about dataset and architecture coverage in our response to Q3.

Response to W2 and Q4: The role of cross-channel predictive relationships

Thank you for suggesting that we examine channel-wise historical input formulation under different cross-channel relationships. We agree that a channel-wise criterion may miss predictive information beyond the target’s own history. We therefore compared independent AR with cross-channel-assisted estimation on ETT and PEMS, retaining the same historical input formulations.

For each channel and horizon, (m,p)(m,p) is determined from the complete training segment only. The assisted estimator uses training-time lagged second-order statistics from all channels to construct auxiliary correlation estimates for the target, followed by coefficient estimation through the Yule–Walker equations. At prediction time, both methods use only the target channel’s history and fixed AR coefficients.

Both methods use the same data splits, complete test forecast origins, and evaluation metrics. All channels are included, and prediction errors are normalized by each channel’s original-training standard deviation.

Internal standardization during fitting differs: independent AR uses the transformed training mean and standard deviation, whereas the assisted version uses mean 0 and scale 1, omitting internal centering and scaling. Table 2 therefore reports the combined effect of cross-channel-assisted estimation and this standardization choice; its full gains cannot be attributed solely to cross-channel information. Final metric normalization is identical.

Table 2. Mean MSE at fixed historical input formulations. Channels and the four horizons H∈96,192,336,720H\in\\{96,192,336,720\\} receive equal weight. Internal fitting standardization differs as described above.

DatasetIndependent AR MSECross-channel-assisted estimation MSEMSE reduction
ETTh10.427540.409314.26%
ETTh20.348610.332534.61%
ETTm10.344290.341390.84%
ETTm20.243100.236152.86%
PEMS080.385770.3064420.56%
PEMS040.171590.1533010.66%

To examine this confound, we also evaluated an independent AR control with the same formulations and no internal centering or scaling. Relative to this control, assisted estimation reduces four-horizon mean MSE from 0.324054 to 0.306437 on PEMS08 (5.436%) and from 0.156828 to 0.153298 on PEMS04 (2.251%). These reductions are computed as one minus the ratio of the respective four-horizon mean MSEs.

PEMS records traffic flow across a road network, where connectivity, traffic propagation, and shared demand may support cross-channel prediction. ETT contains variables with different physical roles and temporal responses. These contexts offer an interpretation of the different gains, not a direct measurement of cross-channel predictive dependence.

The original formulation procedure is reused. Additional cost arises during training, from constructing cross-channel statistics and estimating coefficients. At prediction time, the input channel count and AR order remain unchanged, giving the same asymptotic forecasting complexity as independent AR with the same formulation.

These results show that channel-wise historical input formulation can be combined with cross-channel-assisted estimation. Gains at fixed formulations do not directly quantify how the benefit of formulation itself varies with cross-channel dependence. Because inference still uses only the target’s history, the estimator does not exploit information available exclusively in other channels at the forecast origin.

We will include these results and estimator details in the revised manuscript, clarifying the distinction between channel-wise input formulation and cross-channel information used during estimation. We thank the reviewer for motivating this analysis, which identifies incorporating cross-channel predictive utility directly into historical input formulation as a concrete direction for extension.

Response to W3 and Q1: Joint formulation versus independent combination

Thank you for suggesting a direct comparison with independently configuring and then combining the two input dimensions. We include this control alongside the single-dimension ablations to distinguish the benefit of adapting both dimensions from the additional value of accounting for their coupling.

All four settings use the same training-only criterion Qc,H(m,p)Q_{c,H}(m,p) for channel cc and forecast horizon HH:

  1. Structure only: Configure the temporal structure at fixed extent p=Hp=H:

m^c,HH=arg⁡min⁡m∈MQc,H(m,H),\hat m^{H}_{c,H}=\arg\min_{m\in\mathcal M}Q_{c,H}(m,H),

and use (m^c,HH,H)(\hat m^{H}_{c,H},H).

  1. Extent only: Configure the temporal extent under identity:

p^c,HId=arg⁡min⁡p∈PQc,H(Id,p),\hat p^{\mathrm{Id}}_{c,H}=\arg\min_{p\in\mathcal P}Q_{c,H}(\mathrm{Id},p),

and use (Id,p^c,HId)(\mathrm{Id},\hat p^{\mathrm{Id}}_{c,H}).

  1. Independent combination: Determine both quantities separately as above, then use (m^c,HH,p^c,HId)(\hat m^{H}_{c,H},\hat p^{\mathrm{Id}}_{c,H}). Neither dimension is reconfigured after combination.

  2. Joint formulation: Configure both dimensions together:

(m^c,HJ,p^c,HJ)=arg⁡min⁡m∈M,;p∈PQc,H(m,p).(\hat m^{J}_{c,H},\hat p^{J}_{c,H})=\arg\min_{m\in\mathcal M,\\;p\in\mathcal P}Q_{c,H}(m,p).

The comparison covers seven datasets and H∈96,192,336,720H\in\\{96,192,336,720\\}. Each channel–horizon formulation uses the complete training series, without validation or test outcomes. We hold the scalar linear autoregressive forecaster, data splits, candidate families and extents, reconstruction rules and evaluation protocol fixed.

Table 3. MSE / MAE under four historical input formulations, averaged equally over four horizons. Reconstructed errors are normalized by each channel's original-training standard deviation. Channels are equally weighted within tasks; the overall mean equally weights all 28 dataset–horizon tasks.

DatasetStructure onlyExtent onlyIndependent combinationJoint formulation
ETTh10.4159 / 0.42140.4456 / 0.44790.4337 / 0.43710.4275 / 0.4322
ETTh20.3595 / 0.39640.7684 / 0.57690.3572 / 0.39800.3486 / 0.3982
ETTm10.3606 / 0.38400.3461 / 0.37830.3450 / 0.37820.3443 / 0.3774
ETTm20.2626 / 0.31660.3370 / 0.36990.2447 / 0.31410.2431 / 0.3140
Weather0.2465 / 0.29290.2118 / 0.26560.2181 / 0.26800.2171 / 0.2662
Electricity0.1680 / 0.26060.1644 / 0.26350.1634 / 0.26000.1651 / 0.2617
Traffic0.4607 / 0.29220.4486 / 0.31970.4440 / 0.30690.4486 / 0.3069
Overall mean0.3248 / 0.33770.3888 / 0.37460.3152 / 0.33750.3135 / 0.3367

Adapting both dimensions. Relative to structure only and extent only, joint formulation achieves mean task-wise relative MSE reductions of 4.23% and 10.60%, respectively, with 19 wins and 9 losses in each comparison. These ablations support complementary average benefits, but alone do not establish an advantage over independent combination.

Joint versus independent combination. Independent obtains overall MSE / MAE of 0.3152 / 0.3375, versus 0.3135 / 0.3367 for Joint. Joint's mean paired relative MSE reduction is 0.36%, with 18/0/10 wins/ties/losses and a descriptive 95% task-bootstrap interval of [-0.22%, 0.96%]. The means are slightly lower, but the interval crosses zero; these results do not establish a clear joint-formulation advantage or show that joint search is necessary. All reported percentage reductions are computed within each task before equal averaging, not from ratios of overall MSE means.

The methodological motivation is that temporal structure changes both the dependencies presented to the forecaster and error propagation through reconstruction. An extent configured under identity may therefore differ from one preferred under another representation. Joint evaluation accounts for this interaction within the criterion; that motivation alone does not establish superior forecasting.

We agree that the four representation families and finite extent grid are predefined design choices: a compact, interpretable and reproducible implementation, not an exhaustive characterization of historical inputs. We will clarify their rationale and scope, including representation-specific boundary handling.

We thank the reviewer for motivating this distinction. We will report the independent-combination control and single-dimension ablations together, distinguishing adaptation benefits from the additional value of joint formulation. These comparisons are development-stage results.

Reviewer_YJ1gOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper questions an ubiquitous convention in long-term time-series forecasting: while research focuses overwhelmingly on architectural innovation, the choice of historical temporal extent and temporal structure is almost universally treated as an arbitrary, fixed hyperparameter. The authors reframe historical input selection as an explicit component of the forecasting task formulation.


The input space is defined by reversible temporal structures (identity, first differencing, recurring seasonal template deviation, seasonal differencing) and candidate lookback extents ranging from 96 to 1920 steps.


The authors propose a training-only multi-step risk criterion Q=V+U+DQ = V + U + D. Here, VV evaluates plug-in multi-step forecasting risk on the original data scale via a scalar linear autoregressive reference model solved in closed form with Yule–Walker equations and Levinson recursion; UU penalizes high-order parameter estimation instability via Gaussian perturbations; and DD penalizes persistent level-drift governed by a variance-ratio alternation gate.


Candidate pairs are selected purely from the training series without validation leakage or downstream model re-fitting. The resulting configurations are transferred across multiple downstream models, including scalar AR, DLinear, PatchTST, Hankel-OLS, and projected SVD-Hankel.

Critical strengths

This paper questions an ubiquitous convention in long-term time-series forecasting: while research focuses overwhelmingly on architectural innovation, the choice of historical temporal extent and temporal structure is almost universally treated as an arbitrary, fixed hyperparameter. The authors reframe historical input selection as an explicit component of the forecasting task formulation.


The input space is defined by reversible temporal structures (identity, first differencing, recurring seasonal template deviation, seasonal differencing) and candidate lookback extents ranging from 96 to 1920 steps.


The authors propose a training-only multi-step risk criterion Q=V+U+DQ = V + U + D. Here, VV evaluates plug-in multi-step forecasting risk on the original data scale via a scalar linear autoregressive reference model solved in closed form with Yule–Walker equations and Levinson recursion; UU penalizes high-order parameter estimation instability via Gaussian perturbations; and DD penalizes persistent level-drift governed by a variance-ratio alternation gate.


Candidate pairs are selected purely from the training series without validation leakage or downstream model re-fitting. The resulting configurations are transferred across multiple downstream models, including scalar AR, DLinear, PatchTST, Hankel-OLS, and projected SVD-Hankel.

Critical weaknesses

The criterion QQ relies on a scalar linear AR reference model. While this aligns naturally with linear operator families, it does not explicitly capture the non-linear cross-time and cross-channel attention dynamics learned by deep Transformers.


For models requiring a shared input length across all channels (such as PatchTST and DLinear), reducing channel-specific choices to a single channel-majority structure and median extent discards valuable cross-channel heterogeneity, as observed in the Weather dataset transfer experiments.

Questions for authors

The selection criterion Q=V+U+DQ = V + U + D is derived from a scalar linear autoregressive reference model. In complex deep architectures such as PatchTST or TimesNet, self-attention and non-linear token interactions capture high-order and cross-channel dynamics. Could the authors discuss whether the linear AR proxy could be augmented with a lightweight non-linear proxy without sacrificing closed-form computational efficiency?


For architectures that mandate a shared input specification across all series (such as PatchTST and DLinear), reducing channel-specific choices to a single channel-majority structure and median extent caused a notable performance drop on the Weather dataset (Table 3). Could this limitation be mitigated by channel clustering—grouping channels with similar (m,p)(m, p) choices into shared sub-models—rather than enforcing a single global configuration?


The candidate representation family currently consists of four classical linear and seasonal transformations (identity, first differencing, seasonal template deviation, and seasonal differencing). How easily can the framework incorporate other non-stationary operators, such as fractional differencing, adaptive trend-filtering, or continuous wavelet transformations?


The candidate extent grid contains 39 lookback lengths up to 1920 steps. While Levinson recursion is computationally lightweight, does evaluating all 156 (m,p)(m, p) combinations scale efficiently to industrial datasets comprising tens of thousands of channels (e.g., large-scale sensor or traffic networks)? Can a coarse-to-fine search heuristic safely prune candidate extents?

7 comments

Response to W1: Limitations of the linear AR reference

Thank you for highlighting this limitation. We agree that a scalar AR reference does not explicitly capture the nonlinear temporal interactions or cross-channel dependencies that some downstream Transformers can learn. However, its role is to formulate the historical-input configuration, not to reproduce the downstream predictor’s internal dynamics. The downstream model independently learns its forecasting function from the configured input and retains its nonlinear modeling capacity.

Our original experiments show that AR-based input configuration can benefit a nonlinear Transformer forecaster. In Table 3 of the manuscript, AR-based lookback configuration reduces PatchTST’s four-horizon average MSE from 0.4543 to 0.4221 on ETTh1, from 0.2820 to 0.2624 on ETTm2, and from 0.2579 to 0.2208 on Weather, relative to fixed L=96L=96. A plausible explanation for successful transfer is that linear and nonlinear forecasters can benefit from overlapping predictive information, including persistence, periodicity, and lagged dependencies. An AR reference can identify useful history coverage through these dependencies without modeling every interaction available to a Transformer. Transfer therefore does not require identical predictions or complete agreement in candidate rankings; the formulated configuration can still improve upon a fixed input convention. Nevertheless, the mixed results for full (m,p)(m,p) transfer indicate that representation formulation is more predictor-dependent. We therefore do not claim that the AR criterion yields optimal configurations for every architecture.

We additionally examined alternative configuration references. We evaluated window mean, exponentially weighted mean, and damped Holt alongside AR, transferring their formulated configurations to four downstream forecasters. The evaluation covers ETTh1, ETTm2, and Weather with H∈96,192,336,720H\in\\{96,192,336,720\\}, yielding 12 dataset–horizon tasks.

In this additional experiment, DLinear and PatchTST use channels grouped by formulated representation, with the median formulated lookback within each group. The Hankel forecasters retain channel-specific configurations. MSE is computed after inverse reconstruction, with prediction errors normalized by each channel’s raw-training standard deviation, and averaged equally across the 12 tasks. Bold values indicate the lowest mean MSE in each column.

Table 1. Mean downstream MSE under alternative configuration references.

Configuration referenceDLinear MSEPatchTST MSEHankel-OLS MSESVD-Hankel MSE
AR0.30280.35240.29820.3258
Window mean0.31950.35050.30710.3301
Exponentially weighted mean0.30960.31170.32740.3577
Damped Holt0.34470.30820.43200.4736

We thank the reviewer again for the valuable insight that the suitability of a configuration reference may depend on the downstream predictor. This motivated us to explore alternative lightweight references. In these additional experiments, AR achieves the lowest mean MSE for DLinear and both Hankel forecasters, while exponentially weighted mean and damped Holt yield lower MSE for PatchTST. These findings suggest that reference–predictor compatibility deserves further attention and that AR need not be the preferred reference for every architecture. Although these alternatives do not directly address nonlinear dependency modeling, the results motivate a broader investigation of predictor-aware configuration references.

We will revise the manuscript to make the role and limitations of the AR reference more explicit, distinguishing the observed benefits of lookback transfer from the more predictor-dependent results of full (m,p)(m,p) transfer. Building on the reviewer’s suggestion, we will also discuss lightweight nonlinear and multivariate references as potential extensions, together with the corresponding risk and uncertainty formulations needed to retain a computationally efficient, training-only configuration procedure.

Response to W2 and Q2: Preserving channel heterogeneity through grouped configurations

Thank you for identifying this limitation and suggesting channel grouping. We agree that reducing channel-wise choices to a single majority representation and median lookback can override suitable channel-specific configurations. Requiring a shared input length does not, by itself, require identical preprocessing across channels. For example, at longer Weather horizons, precipitation channels originally assigned the identity representation were instead subjected to seasonal differencing under the global configuration. Such reassignment may be unfavorable because inverse differencing can accumulate forecast bias and correlated errors.

Motivated by the reviewer’s suggestion, we evaluated a structure-based grouping strategy. Channels were partitioned into identity, first-difference, template-removal, and seasonal-difference groups according to their training-selected mcm_c. Each group received a separately trained forecaster using its corresponding preprocessing. In this experiment, all groups used a common lookback of p=1776p=1776, rather than the group-wise median lookbacks used in the reference-comparison experiment reported in our response to W1.

Group assignments and preprocessing choices used training information only. We evaluated all four Weather horizons, H∈96,192,336,720H\in\\{96,192,336,720\\}, retaining the original data splits, training recipe, and original-scale evaluation metrics. The table reports four-horizon mean test MSE.

Table 2. Four-horizon mean Weather test MSE under grouped configurations.

Input specificationModels per taskDLinearPatchTST
Configured lookback pp only, identity input10.23500.2208
Globally shared full configuration (m,p)(m,p)10.65630.6729
New: grouped by mcm_c, with corresponding preprocessing and p=1776p=177640.22860.2722

The grouped configuration reduces mean MSE from 0.6563 to 0.2286 for DLinear and from 0.6729 to 0.2722 for PatchTST, substantially mitigating the degradation observed under global sharing. This improvement comes at the cost of training four separate sub-models. Because the experiment also changes lookback, parameter sharing, and total model capacity, we interpret it as evidence for the practical value of the grouped adaptation strategy, rather than an isolated estimate of the effect of grouping alone.

The results also reveal that the benefit of grouping varies across downstream architectures. Grouped DLinear achieves slightly lower mean MSE than the lookback-only identity baseline, whereas grouped PatchTST remains worse than that baseline. Thus, preserving channel-specific temporal structures substantially mitigates the degradation caused by global sharing, but the tested grouping strategy does not fully recover the lookback-only baseline performance for PatchTST. Grouping by temporal structure alone still pools channels with potentially different temporal extents and dynamics, leaving room for more refined historical-input formulations.

We will include these results and their additional computational cost in the revised manuscript. We will also distinguish the tested strategy—grouping by representation with a common lookback—from more general clustering over both (m,p)(m,p), including grouping under a fixed model-count budget. The reviewer’s suggestion has helped us identify a concrete mitigation and clarify its remaining limitations.

Response to Q1: Lightweight nonlinear augmentation and closed-form efficiency

Thank you for this constructive suggestion. It motivated us to investigate a lightweight nonlinear augmentation and measure its accuracy–computation trade-off. The tested design retains analytic estimation of the AR components, but introduces an optimized neural gate; it therefore does not preserve a fully closed-form fitting procedure.

Specifically, we retain the complete pp-order AR predictor and augment it with a full-order ridge-AR correction. An 8–16–4 MLP with 212 trainable parameters uses features of the current history to gate four lag-group contributions from this correction. The augmented predictor contains 2p2p AR coefficients and 212 neural parameters.

All fitting uses the complete training segment. For this experiment, we hold the historical-input formulations derived from the original QQ fixed. This allows us to examine nonlinear augmentation at the configured inputs. The results are shown in Table 3.

Table 3. Forecasting accuracy on the five-dataset development panel, including all channels. Each dataset entry averages four forecast horizons; Overall equally weights the 20 dataset–horizon tasks.

DatasetChannelsOriginal AR MSEAR + neural correction MSEMSE reduction
ETTh270.3486090.3458610.79%
ETTm170.3442920.3425750.50%
Weather210.2170620.2162750.36%
Electricity3210.1651340.1644650.41%
Traffic8620.4486290.4398741.95%
Overall1,2180.3047450.3018100.96%

The augmentation reduces MSE on 18 of the 20 dataset–horizon tasks, with a 0.96% reduction in the overall mean MSE. These results suggest modest predictive benefits from the nonlinear augmentation at the existing input formulations.

We also measured fitting cost on ETTh2/HUFL using all 8,640 training observations, one CPU core, single-threaded Torch/BLAS, and no GPU. For each horizon, the original and augmented fits were interleaved in the order ABBAABBA, retaining all four repetitions per method.

Table 4. Fitting time at fixed historical-input formulations. Times are in seconds. Additional fitting time is the median of the four paired differences.

Horizon HHConfigured extent ppAR fit: medianAugmented fit: medianAdditional fitting time
961,1520.0564.9604.904
1924800.0285.2705.242
3364800.0316.0766.045
7206720.0539.9119.858

Both timings cover preprocessing through prediction-operator construction at the fixed input formulations. The augmented fit includes the additional ridge AR, risk-moment preparation, initialization-gate training, and final neural-gate optimization. Configuration search, data loading, library imports, and artifact writing are excluded. Optimization of the final 212-parameter gate alone takes median times of 1.869–2.161 seconds.

The additional fitting cost is approximately 5–10 seconds in this single-channel CPU experiment. Although limited in absolute duration, it is substantial relative to the original AR fit. Moreover, these measurements do not establish the cost of evaluating the augmented reference across all candidate historical-input formulations.

To more closely preserve closed-form fitting, a potential alternative is to use fixed causal nonlinear features with a ridge-fitted output head. Such a predictor can be nonlinear in the historical observations while retaining a closed-form solution for the output coefficients. However, closed-form coefficient fitting alone does not guarantee that the reconstructed multi-step risk and uncertainty terms remain analytically tractable; these would require corresponding derivations.

We will include this exploratory result and clarify the distinction between nonlinear prediction at fixed input formulations and nonlinear-reference-based input formulation. We thank the reviewer for this suggestion, which motivates further investigation of the latter, jointly considering formulation quality, forecasting accuracy, and the computational cost of the complete configuration procedure.

Reviewer_TAUuOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes configuring historical inputs before fitting a forecasting model. For each channel and forecast horizon, it selects both a temporal representation and a historical length. The four candidate representations are the original series, first differences, deviations from a learned seasonal template, and seasonal differences. Experiments demonstrate substantial improvements over several fixed-input configurations and provide useful controls against alternative length selectors.

Critical strengths
  1. The paper addresses an interesting practical choice in forecasting. The proposal to make these decisions before fitting an expensive downstream model has practical appeal, particularly when architecture-specific input search would require many training runs.

  2. Comparing reconstructed forecasts on the original scale is a sound methodological choice.

  3. The same-model experiments provide meaningful evidence that the proposed configuration helps. The experiments are comprehensive and solid.

Critical weaknesses
  1. Tab 3 shows that full configuration worsens Weather MSE from 0.2350 to 0.6563 for DLinear and 0.2208 to 0.6729 for PatchTST. Can the author explain the reasons?

  2. Tab 1 caption states that all baselines use 96 input steps, while Section 4.1 allows the proposed method up to 1920. Is this a fair comparison?

  3. Sec 2.3 already discusses earlier multi-step autoregressive order-selection methods. Eq.2 uses similar representations: levels, first differences, seasonal residuals, and seasonal differences. Can the author explain more precisely what the new scoring rule adds beyond these established ingredients?

Questions for authors

See weakness. I might adjust my scores based on the authors responses.

5 comments

Response to W1: Explaining the Weather transfer degradation

Thank you for drawing our attention to this substantial degradation. We agree that it warrants a clearer explanation. Our examination suggests that imposing a single temporal structure on heterogeneous Weather channels is a plausible contributor. Motivated by this observation, we evaluated a grouped adaptation that preserves channel-specific temporal structures. This substantially reduces the errors for both DLinear and PatchTST, although PatchTST does not recover the performance of its extent-only baseline.

In the original transfer implementation, the modal temporal-structure family and median lookback are derived from channel-wise historical input formulations and then applied to all channels. A shared input length does not itself require identical preprocessing, and this aggregation can override suitable channel-specific structures. For example, at longer forecasting horizons, precipitation channels originally associated with the identity representation are subjected to seasonal differencing. Inverse seasonal differencing reconstructs forecasts by accumulating predicted changes along seasonal lags, so forecast bias and correlated errors can accumulate when this representation is poorly suited to a channel or predictor.

To examine whether preserving temporal-structure heterogeneity mitigates this degradation, we conducted grouped adaptation experiments on all four Weather horizons, H∈96,192,336,720H\in\\{96,192,336,720\\}. Channels were partitioned into four groups according to their training-derived temporal structures mcm_c. Each group received a separately trained forecaster with its corresponding preprocessing, and all groups used a common lookback of p=1776p=1776. We retained the original data splits, training recipe, and original-scale evaluation metrics.

Table 1. Four-horizon mean test MSE for global and grouped historical input formulations on Weather.

Input formulationModels per taskDLinearPatchTST
Fixed L=96L=9610.26700.2579
Fixed L=HL=H10.25000.2333
Extent-only formulation (pp), identity input10.23500.2208
Globally shared full formulation (m,p)(m,p)10.65630.6729
grouped by mcm_c, with corresponding preprocessing and p=1776p=177640.22860.2722

The grouped alternative reduces mean MSE from 0.6563 to 0.2286 for DLinear and from 0.6729 to 0.2722 for PatchTST. These results are consistent with preserving channel heterogeneity helping to mitigate the degradation. Because lookback, parameter sharing, and model capacity also change, the comparison does not isolate the contribution of the aggregation rule. The additional cost is training and maintaining four separate sub-models per task.

The results also show that grouping does not provide a net benefit for every predictor. Grouped DLinear achieves slightly lower MSE than its extent-only identity baseline, whereas grouped PatchTST remains worse. Thus, preserving channel-specific structures substantially mitigates the observed failure but does not fully resolve it. The suitability of a temporal representation can still depend on the downstream predictor, including how it forecasts transformed values and how its errors propagate through reconstruction.

We will include these results and their additional cost, explain the limitations of global structure aggregation, and qualify the scope of full-formulation transfer. We thank the reviewer for motivating this examination and an alternative adaptation that better preserves channel-specific temporal structure.

Response to W2: Fairness of the historical input comparison

Thank you for raising this important concern. We agree that manuscript Table 1, with 96-step baselines and larger candidate extents for our method, does not isolate historical input formulation under equal budgets. We therefore added a direct comparison under the same maximum raw-history budget.

We trained DLinear and PatchTST with L=1920 on ETTh1 and ETTm1 using their released dataset-specific recipes. They received the original series with their standard normalization and internal components; our input transformations were not applied. Our AR formulations used the complete training segment only, with the raw span, including differencing anchors, capped at 1920 observations. All methods used the same splits, complete test forecast origins and metric normalization. Neural checkpoints were selected using validation data.

Table 2. MSE / MAE under the 1920-observation raw-history cap. “Average” equally weights H=96,192,336,720.

DatasetHorizon HHDLinear, L=1920L=1920PatchTST, L=1920L=1920Our configured AR
ETTh1960.3942 / 0.41840.3813 / 0.41590.4022 / 0.4124
ETTh11920.4384 / 0.45060.4166 / 0.43770.4289 / 0.4299
ETTh13360.4729 / 0.47550.4692 / 0.48100.4422 / 0.4380
ETTh17200.5282 / 0.53240.4894 / 0.49730.4369 / 0.4487
ETTh1Average0.4584 / 0.46920.4391 / 0.45800.4275 / 0.4322
ETTm1960.3087 / 0.35550.3053 / 0.35960.2939 / 0.3464
ETTm11920.3350 / 0.37120.3340 / 0.37740.3304 / 0.3680
ETTm13360.3823 / 0.41480.3579 / 0.39340.3580 / 0.3862
ETTm17200.4041 / 0.41680.4079 / 0.42200.3948 / 0.4089
ETTm1Average0.3575 / 0.38960.3513 / 0.38810.3443 / 0.3774

Our AR achieves lower MSE in 7/8 tasks versus DLinear and 5/8 versus PatchTST, and lower MAE than both in all eight tasks. Its mean MSE and MAE are lower on both datasets. Thus, the observed advantage cannot be explained solely by the original input-length difference.

To examine temporal extent more broadly, we completed 320 runs: two models, two datasets, four horizons and 20 lookback lengths from 96 to 1920 in steps of 96, using seed 2021. Within each task, only the lookback length varied. All 16 MSE sequences were non-monotonic, although L=1920 improved on L=96 in 14/16. The matched-budget comparison addresses fairness directly; the sweep supplies complementary context.

Table 3. ETTm1 MSE sweep summary.

ModelHL=96L=1920Minimum (L)
DLinear960.34560.30870.2993 (288)
DLinear1920.38220.33500.3350 (1920)
DLinear3360.41400.38230.3584 (1536)
DLinear7200.47530.40410.3989 (1440)
PatchTST960.32300.30530.2880 (672)
PatchTST1920.35950.33400.3277 (192)
PatchTST3360.38970.35790.3579 (1920)
PatchTST7200.45380.40790.4002 (1632)

Table 4. ETTh1 MAE sweep summary.

ModelHL=96L=1920Minimum (L)
DLinear960.39610.41840.3928 (384)
DLinear1920.44410.45060.4155 (288)
DLinear3360.45540.47550.4390 (480)
DLinear7200.53170.53240.4987 (288)
PatchTST960.40570.41590.3984 (384)
PatchTST1920.43340.43770.4201 (288)
PatchTST3360.45500.48100.4340 (288)
PatchTST7200.47580.49730.4641 (288)

Tables 3–4 summarize the endpoints and post-hoc minima of the two original plots. Minima were determined from unrounded test errors and were never used for formulation or model tuning. Fixed L=1920 need not be the best neural extent, so we do not claim superiority over architecture-specific, validation-tuned histories. The same-forecaster ablations in our response to W3 examine the formulation criterion separately from architectural differences. We will clarify the budgets and the role of each comparison in the revised caption and discussion. Thank you for motivating this additional evaluation.

Response to W3: Contribution beyond classical representations and AR predictor selection

Thank you for asking us to clarify what the scoring rule adds beyond these established ingredients. We agree that the transformations and forecast-oriented AR selection have substantial foundations. Ing (2004) [1] studies prediction orders and recursive and direct prediction methods; Ing et al. (2009) [2] further characterize multi-step prediction risk and predictor selection in nonstationary autoregressions. Our work explicitly formulates the historical input configuration problem and provides a concrete training-only solution.

1. From AR predictor configuration to historical input formulation

We formulate temporal structure and temporal extent together. Changing temporal structure changes the quantity presented to the predictor and the mapping needed to reconstruct its forecasts. Our criterion compares candidate (m,p)(m,p) formulations against the same original-scale, multi-step forecasting target while accounting for these mappings.

For example, with first differences rt=xt−xt−1r_t=x_t-x_{t-1}, reconstruction gives

xt+h−x^t+h=∑j=1h(rt+j−r^t+j).x_{t+h}-\widehat{x}_{t+h}=\sum_{j=1}^{h}\left(r_{t+j}-\widehat r_{t+j}\right).

The original-series squared error therefore contains both squared forecast-step errors and cross-step terms. Seasonal differencing accumulates errors along seasonal chains, whereas identity and fixed-template residuals follow different reconstruction rules. Incorporating these rules permits a common risk comparison across structures and extents, rather than configuring AR order within a prescribed representation.

2. A concrete scoring solution

The criterion Q=V+U+DQ=V+U+D combines three components:

  • Reconstructed multi-step risk, VV. Each reconstructed error is expressed through an aggregate future target and its recursive prediction from the historical state. Training autocovariances supply the target variance, target–history covariance and history covariance for a plug-in risk estimate under the working model. This includes reconstruction-induced cross-step terms on the original-series squared-error scale.
  • Propagated coefficient uncertainty, UU. Perturbations drawn using the estimated coefficient covariance are propagated through recursive forecasting and reconstruction. This approximates their contribution to original-series forecasting risk, accounting for temporal structure, extent and horizon rather than penalizing model size alone.
  • Persistent level variation, DD. This empirical, training-derived adjustment uses local-level variation, a non-alternation gate and representation-dependent weights. For fixed mm, it is independent of pp: it adjusts comparisons between temporal structures rather than adding another order-complexity penalty.

Candidates are evaluated using the complete training segment only. Historical inputs are fixed before downstream fitting, so the downstream forecaster is fitted at the chosen input rather than repeatedly trained for every candidate.

3. Controlled evidence

We hold the scalar linear autoregressive forecaster, candidate representations, temporal extent grid, data splits, reconstruction and evaluation protocol fixed.

Table 5. Scoring-criterion ablations, averaged equally over seven datasets and four forecast horizons.

Historical input formulation criterionMSEMAE
VV0.41510.3851
V+UV+U0.40520.3746
V+DV+D0.32700.3492
Full Q=V+U+DQ=V+U+D0.31350.3367

The full criterion achieves the lowest mean MSE and MAE. Removing UU increases mean MSE from 0.3135 to 0.3270; removing DD increases it to 0.4052. These results support the complete scoring construction and its use in training-only historical input formulation. They do not establish a universal ranking guarantee for downstream architectures: VV is a working-model estimate, UU is approximate and DD is empirical.

We will revise Section 2.3 and the method discussion to clarify the relationship to prior statistical work and the specific solution developed here. We thank the reviewer for prompting a clearer account of this contribution.

References

[1] Ching-Kang Ing. Selecting Optimal Multistep Predictors for Autoregressive Processes of Unknown Order. The Annals of Statistics, 32(2): 693–722, 2004.

[2] Ching-Kang Ing, Jin-Lung Lin, and Shu-Hui Yu. Toward Optimal Multistep Forecasts in Non-stationary Autoregressions. Bernoulli, 15(2): 402–437, 2009.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.