acceptodds
Under review as a conference paper at ICLR 2027

CrossForest: Phase-Specialized Learners for Long-Horizon Time Series Forecasting

Abstract

Time series forecasting supports applications such as traffic planning, energy management and weather forecasting. Forecast accuracy often declines as the prediction horizon increases. To take advantage of the higher accuracy often seen at shorter prediction lengths while preserving the full forecast horizon, we propose CrossForest, a framework that can be applied to different forecasting backbones. It splits a long forecast into phase-specific tasks aligned to a common time origin. For phases, it routes each timestamp by its residue modulo to a phase learner with its own parameters. Each learner predicts only its assigned phase: values spaced time steps apart. Cross-interleaving assembles these outputs into the complete -step forecast. Adjacent sliding windows share most phase tasks, so the training set keeps one copy of each distinct task and retains complete stride-one coverage without repeated computation. We call the combination of globally aligned phase assignment and duplicate-free task construction the Global-Unique protocol. Our experiments show that WPMixer and PatchTST with CrossForest outperform the baselines on multiple forecasting tasks, while CrossForest improves eight of the nine baseline backbones.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, long-horizon forecasting, phase-specific learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Hp3fOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies long-horizon time series forecasting, where a model takes a look-back window of length LL and outputs the next HH steps. The authors propose CrossForest, which splits the HH-step horizon into NN interleaved sub-sequences by phase, so that learner rr is responsible for the timestamps whose absolute index is congruent to r mod Nr \bmod N, and each learner owns its own parameters. Two design choices are emphasized: phase labels are assigned by the absolute timestamp index rather than by the position inside a sliding window, and under this global routing every timestamp is consumed exactly once, which removes the duplicate consumption of the window-local protocol. The framework is backbone-agnostic and is instantiated on nine LTSF models. Across 324 matched comparisons the macro-average MSE/MAE drop by 7.84%/4.76%, with 8 of 9 backbones improving on both metrics.

Critical strengths
  • A broad, systematic empirical study. 324 matched comparisons (nine backbones × nine datasets × four horizons), and an explicit channel-count and phase-count sensitivity sweep, is more empirical work than is typical, and the sensitivity analysis is honest — the authors report that the benefit shrinks with the number of channels and turns negative on Traffic, rather than hiding it.
  • Simple, cheap, and backbone-agnostic. The framework requires no change to the underlying architecture and adds only routing/aggregation, which is attractive for practitioners who already have a tuned backbone. The global-routing + dedup protocol is a clean fix to a real inefficiency of window-local phase assignment.
Critical weaknesses
  • I am not fully convinced that the phase decomposition is new enough. Splitting the horizon into interleaved sub-sequences and giving each sub-sequence a separate model is very close to SutraNets, which also assigns a separate sequence model to every KK-th value, and SparseTSF is the shared-parameter version of the same construction. My understanding is that the two advertised contributions are a routing convention (phase labels from the absolute index rather than the window position) and a computational saving (dedup), rather than a new modelling principle. It would help if the paper could state more explicitly what it adds beyond this literature.
  • I am also not able to tell how much of the gain comes from the method itself and how much from the extra parameters. Each phase learner has its own parameters, so the NN-phase model has roughly NN times the parameters of the N=1N=1 control, and no parameter-matched control is reported. The closest available control, independent vs. shared phase parameters, wins only 25/36 comparisons, which, at least to me, suggests that specialization alone contributes little. A single N=1N=1 learner widened to the same total parameter count as the NN-phase model would go a long way toward separating the two effects, and I would encourage the authors to add it.
Questions for authors

1, Please report a parameter-matched control: a single learner with the same total parameter count as the NN-phase CrossForest. How much of the gain survives? 2, Given Figure 2, under what conditions do you recommend CrossForest? Is there a principled rule (in terms of channel count, horizon, or periodicity) that predicts when it will help?

1 comment

Thank you for raising this point. We added a parameter-matched experiment in Section 4.4.1, “Parameter Sharing and Parameter-Matched Controls.”

We adjusted the model width dmodel of the single-learner control so that its total trainable parameter count was within 2% of that of the corresponding CrossForest model. In this experiment, CrossForest consistently achieved lower MSE and MAE than the parameter-matched single learner. These results indicate that the increase in parameter count alone does not explain CrossForest’s gains.

At present, we can identify only one clear limitation: CrossForest’s performance deteriorates substantially on high-dimensional data with many channels.

Reviewer_RHuFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CrossForest wraps a forecasting backbone with parameter-disjoint phase learners. Absolute timestamp residues modulo N determine each learner’s subsampled history and future targets; interleaving reconstructs the full forecast. The Global-Unique protocol combines consistent phase assignments across windows/splits with duplicate-free task construction. Under the stated divisibility and contiguous-window assumptions, W stride-one windows yield W+N−1 distinct phase tasks instead of WN. Experiments compare nine backbones over nine datasets and four horizons, using validation-selected phase counts, and study parameter sharing, training protocols, and channel-count sensitivity.

Critical strengths
  1. The task organization is precise and has a concrete computational motivation. Global alignment gives each parameter set a consistent role, including at nonzero split offsets. The duplicate-task argument explains a real redundancy introduced by overlapping windows after phase factorization, and the construction preserves coverage of distinct supervision pairs. This is a useful implementation-level insight, even though fewer task instances do not by themselves prove an end-to-end speedup over an unfactorized model.

  2. The evaluation tests generality and reports important failure cases. Table 2 compares matched configurations across nine backbones and reports improvement for eight, rather than relying on one favorable model. Validation-based phase selection and the reported DLinear degradation are useful controls. Figure 2 and the sharing/protocol ablations further show where gains weaken or reverse. Reporting these negative results makes the actual scope of the method more assessable.

Critical weaknesses
  1. The evidence leaves the useful operating regime insufficiently characterized. The Electricity and Traffic comparisons in the main cross-backbone grid use fixed Last7 subsets. In the channel-count analysis, PatchTST’s average Electricity MSE improvement changes from +14.15% at seven channels to −4.98% with all channels; WPMixer also becomes worse on the full set. The paper acknowledges this limitation, so it is not a disclosure issue. The unresolved question is what reliable, generalizable advantage remains and when phase isolation is appropriate. Shorter output sequences still span the full physical horizon and exclude other phases’ histories from each learner, so sequence-length reduction alone does not explain the observed accuracy changes.

  2. The ablations do not isolate the contributions under comparable optimization and capacity budgets. Global-Unique versus WindowLocal-Repeated changes both phase alignment and task repetition. Independent versus shared learners changes parameterization and total capacity. Although these comparisons are informative, they cannot separately attribute gains to global alignment, deduplication, and phase specialization. Deduplication also changes the number of updates per epoch: Appendix D.2 uses identical backbone hyperparameters for both protocols, including the fixed epoch and patience limits in Table 7, so WindowLocal-Repeated trains on roughly N times as many task instances per epoch as Global-Unique, and equal epoch limits do not imply equal optimization budgets. In addition, the results appear to come from single training runs; I could not find seed repetitions or variability estimates, although several protocol- and sharing-level differences are small (e.g., an average MSE reduction of 0.93% for WPMixer in Table 4). This makes the framework’s incremental benefit over simpler phase-factorized alternatives difficult to establish.

Questions for authors
  1. Clarification of scope: What evidence identifies when phase specialization helps, beyond the number of channels in the current examples? Does a narrower claim focused on the demonstrated settings better describe the contribution, and how should the full-channel regressions affect model selection?

  2. Targeted ablation: Can you provide a globally aligned repeated-task control against Global-Unique, with actual optimizer-update counts and training time reported? Are the reported results from single runs, and if so, how large is the seed-to-seed variability for representative tasks? Together with parameter counts for shared and independent learners, this would help distinguish alignment, deduplication, and capacity effects without requiring a new benchmark suite.

1 comment

Response to Reviewer R2 (RHuF)

We sincerely thank the reviewer for the constructive feedback.

Q1:

Response:

Thank you for your thoughtful feedback on the scope of our work.

1. Beyond channel count, what evidence shows that phase specialization helps?

We added a cross-phase diagnostic in Section 4.4.2. For each target phase, we hold its historical input, time covariates, and forecast targets fixed and have every phase learner in the same trained model make a prediction. We then compare the error of the originally assigned learner with the average error across all learners. The results show that phase learners perform better on their originally assigned phases. Appendix D.3 provides the detailed results.

2. How should the paper's contribution be described with a narrower conclusion?

In the conclusion, we explicitly state that DLinear's average test error increases when combined with CrossForest and that forecasting gains vary with the backbone, phase count, and channel setting. We therefore present CrossForest as a framework that can be combined with existing time-series forecasting models. It can improve forecasting accuracy on low-dimensional data, which gives it practical value.

3. How should the performance degradation in full-channel settings affect model selection?

The full-channel results indicate that model selection should be based on the channel set used in practice. In engineering applications, correlation analysis can guide feature selection and reduce the number of input variables for time-series forecasting models. CrossForest remains worth trying on tasks where feature selection has reduced the input dimensionality.

Q2:

Response:

  1. Thank you for pointing this out. We added Global-Repeated (GR) as a control in Section 4.4.3, Phase Alignment and Training-Task Construction.

    As shown in Section 4.4.3, switching from WLR to GR increased test MSE and MAE on average for both backbones. Compared with GR, GU reduced MSE and MAE by 0.991% and 0.617% on average for WPMixer, and by 5.276% and 3.764% for PatchTST, respectively. Compared with WLR, GU also improved both metrics on average for both backbones. Appendix D.4 provides the experimental settings and full results.

    In these experiments, GU combined global phase alignment with duplicate-task removal and achieved lower average forecast errors than GR and WLR.

  2. All reported evaluation metrics are averaged over runs with five random seeds, and the results vary little across seeds.The specific setup is reflected in the code we submitted.

  3. We added a parameter-matched experiment in Section 4.4.1, Parameter Sharing and Parameter-Matched Controls. The results indicate that an increase in total parameter count alone cannot explain the observed gains from CrossForest.

Reviewer_Na4tOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors focus on the observation that forecasting accuracy generally deteriorates as the prediction horizon increases. To address this issue, the paper proposes CrossForest, which decomposes a long forecasting task into several phase-specific subtasks. Each learner processes observations assigned to one temporal phase and predicts a shorter phase-specific sequence, after which the outputs are interleaved to reconstruct the complete forecast. The paper further introduces the Global-Unique protocol, which maintains globally consistent phase assignments and removes duplicated phase tasks produced by overlapping sliding windows. The reported results are generally promising: CrossForest improves eight of the nine examined forecasting backbones and achieves average reductions of 7.84% in MSE and 4.76% in MAE across the reported comparisons. Nevertheless, I have several concerns regarding the motivation, comparison fairness, computational cost, and the actual generality of the proposed framework.

Critical strengths
  1. The overall framework is simple, intuitive, and compatible with various forecasting backbones. 2) The globally aligned phase assignment avoids inconsistent routing between training, validation, and testing. 3) The duplicate-free task construction substantially reduces redundant training tasks generated by overlapping windows. 4) The experiments cover nine datasets, nine forecasting backbones, and four prediction horizons. 5) The paper includes useful ablations on parameter sharing, phase count, channel count, and training-task construction.
Critical weaknesses
  1. The main experimental results are not sufficiently convincing. Table 1 reports only the averaged results across the four ETT datasets and further averages the performance over four prediction horizons. Although the complete results are included in the appendix, the main paper does not clearly present the performance of each method on individual datasets and prediction horizons. Moreover, CrossForest shows only limited and inconsistent improvements over the strongest competing methods in the main comparison.
  2. The mathematical notation should be presented more rigorously and consistently. For example, the paper defines xt∈RCx_t\in\mathbb{R}^{C}, but subsequently writes Xs=[xs,xs+1,…,xs+L−1]∈RL×CX_s=[x_s,x_{s+1},\ldots,x_{s+L-1}]\in\mathbb{R}^{L\times C}. Under the conventional column-vector notation, this concatenation would instead produce a matrix in RC×L\mathbb{R}^{C\times L}. The authors should explicitly specify whether xtx_t is treated as a row or column vector and revise the matrix construction accordingly. Similar notation should be checked throughout the paper.
  3. The units and interpretation of the time-indexing notation could also be explained more explicitly.
  4. CrossForest employs multiple phase learners with independent parameters. The resulting improvements may come from increased model capacity rather than phase specialization, but the paper does not provide parameter-matched baselines.
  5. The claim of broad applicability is weakened by the fact that DLinear deteriorates and the method performs poorly in many high-channel settings.
  6. The paper does not provide sufficient evidence that different temporal phases genuinely require different forecasting mappings. The improvements of independent learners may simply result from additional parameters.
  7. The comparison between Global-Unique and WindowLocal-Repeated changes both the phase-alignment strategy and duplicate-removal strategy simultaneously. Consequently, the contribution of each component cannot be isolated.
  8. The main cross-backbone evaluation uses only seven channels from Electricity and Traffic. The channel-sensitivity experiment shows that the gains substantially decrease and often become negative when the complete high-dimensional datasets are used.
  9. The proposed framework discards interactions between adjacent temporal phases because each learner observes only its own subsampled sequence. The effect of this information loss on forecast continuity and local dynamics is not sufficiently examined.
Questions for authors

As stated in the weaknesses section.

1 comment

Response to Reviewer R1

Thank you for your constructive feedback. We summarize the revisions below.

Q1

Table 1 now reports MSE and MAE separately for every configuration, dataset, and prediction horizon, so all 36 tasks can be compared in the main text. We agree that the gains are modest: CrossForest improves results on most tasks, but not all. The framework can also be evaluated with new forecasting backbones as they become available.

Q2

Section 3.1 now defines each observation as a column vector xt∈RC×1\boldsymbol{x}_t\in\mathbb{R}^{C\times1}. The input and target windows are L×CL\times C and H×CH\times C, respectively, with time steps in rows and channels in columns. Section 3.2.1 rewrites phase extraction and interleaving using zero-based row indexing: A[j,:]\boldsymbol{A}[j,:] denotes row jj. A phase history has shape (L/N)×C(L/N)\times C, and a phase target or prediction has shape (H/N)×C(H/N)\times C. Reconstruction is explicit: Y^s[k,:]=Y^s(rk)[jk,:]\widehat{\boldsymbol{Y}}_s[k,:]=\widehat{\boldsymbol{Y}}_s^{(r_k)}[j_k,:]. We updated the nonzero starting-phase example and Figure 1 accordingly.

Q3

The abstract now states that each observation is assigned to a phase by its global sample index modulo NN. Section 3.1 defines tt as the zero-based global index of a retained row and ss as the global index of the first historical observation in a window. On the nominal grid, τt=τ0+tΔ\tau_t=\tau_0+t\Delta, and an LL-sample history spans (L−1)Δ(L-1)\Delta from first to last observation.

The ETTm example in Section 3.2.1 uses Δ=15\Delta=15 minutes, N=4N=4, and H=96H=96. Each learner produces 24 hourly predictions; interleaving yields 96 predictions at 15-minute intervals, ending 24 hours after the last historical observation. Figure 1 distinguishes the original interval Δ\Delta from the within-phase interval NΔN\Delta.

Q4

In Section 4.4.1, we adjusted the single learner's width so that its trainable parameter count was within 2% of the corresponding CrossForest model's. CrossForest achieved lower MSE and MAE in all eight settings, suggesting that parameter count alone does not explain the gains. Appendix D.2 gives the configurations and exact counts.

Q5

We narrowed our applicability claims. Section 4.2 reports that DLinear's selected N=2N=2 configuration lowered mean validation MSE by 0.14%, but raised mean test MSE by 0.45% and MAE by 0.21%. The conclusion now explicitly notes DLinear's average deterioration. Section 4.3.1 reports that across 16 full-channel Electricity and Traffic comparisons against N=1N=1, CrossForest increased MSE in 15 and MAE in all 16.

Q6

In the new diagnostic (Section 4.4.2), every phase learner in the same trained model forecasts the same task with identical historical inputs, time covariates, and targets. The learner originally assigned to the phase had lower errors than the average across all learners in some settings. Appendix D.3 gives the full results.

Q7

Section 4.4.3 adds Global-Repeated (GR), which assigns phases by global index and retains repeated tasks from overlapping windows. WindowLocal-Repeated (WLR) also retains repeats but assigns phases by position within each window. Global-Unique (GU) uses global phases and keeps one copy of each distinct input–target pair per learner.

GR increased mean test MSE and MAE relative to WLR for both backbones. Relative to GR, GU reduced mean MSE and MAE by 0.991% and 0.617% for WPMixer and by 5.276% and 3.764% for PatchTST, respectively. GU also improved both metrics on average relative to WLR for both backbones. Appendix D.4 gives the settings and full results.

Q8

We agree that CrossForest has limitations when applied to all channels of high-dimensional datasets. It improved forecasts in some of the lower-dimensional settings we tested. Whether feature selection can help in the full-channel settings remains to be tested.

Q9

Appendix F assesses forecast continuity using the mean squared difference between predicted and observed increments at lags of one to four steps. Against N=1N=1, CrossForest's increment error was 58.03% higher for WPMixer and 21.47% higher for PatchTST at lag one; at lag four, the gaps narrowed to 1.27% and 10.82%, respectively. Yet CrossForest reduced mean pointwise MSE by 4.79% and 6.25%, respectively. Exponential moving average smoothing reduced mean adjacent-step increment error by about 18% for both backbones, but increased mean pointwise MSE and MAE.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.