acceptodds
Under review as a conference paper at ICLR 2027

Beyond Model Libraries: Learning to Compose Forecasting Architectures for Heterogeneous Time Series

Abstract

Time-series forecasting models often exhibit heterogeneous performance across series with diverse temporal patterns, making it challenging to identify a universally optimal architecture. Existing approaches either select from predefined model libraries or rely on large-scale pretrained models, but they are limited by restricted structural coverage, high computational cost, or insufficient adaptation to individual series. In this paper, we propose **RankTS**, a component-level model selection framework that learns architectural preferences from historical forecasting tasks and transfers them to unseen series. RankTS constructs a diverse compositional search space from forecasting operators, explores effective architectures through task-local evolution, and learns a series-conditioned ranker from sparse architecture evaluations. By jointly modeling time-series characteristics and architecture configurations, RankTS efficiently generates and selects suitable forecasting architectures without exhaustive search on each target series. Extensive experiments on five time-series datasets and four forecasting horizons demonstrate that RankTS consistently outperforms existing model selection strategies and achieves competitive performance against strong forecasting model families with limited target-side training.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Architecture Recommendation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gJXyOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents RankTS, a component-level framework for time-series forecasting architecture composition and selection. RankTS first constructs a pool of forecasting components and composes them into candidate architectures represented as directed acyclic graphs (DAGs). It then employs task-local evolution to discover effective architectures on historical time-series tasks. By jointly encoding time-series characteristics and architecture structures, RankTS learns architecture preferences from pairwise historical evaluations, enabling series-conditioned architecture recommendations. Experiments on five benchmark datasets across four forecasting horizons show that RankTS achieves the best forecasting performance in more than half of the evaluated settings, while requiring only three candidate model fits on each target series.

Critical strengths
  1. Novel perspective on forecasting architecture selection. The proposed framework goes beyond conventional model selection and ensembling by exploring component-level architecture composition and recommendation. This offers an interesting and relatively underexplored direction for adapting forecasting architectures to heterogeneous time series.

  2. Effective task-local architecture evolution. The task-local evolution mechanism iteratively generates new candidate architectures based on retained promising ones, enabling the discovery of high-performing architectures tailored to different historical tasks. The experimental results provide evidence of its effectiveness in identifying competitive candidates.

Critical weaknesses
  1. Limited cross-dataset transferability. The results in Figure 7 suggest that the ranker's transferability across datasets remains limited. This raises concerns about the framework's generalization to new datasets, particularly when dataset-specific offline architecture search and ranker training are required. Incorporating cross-dataset adaptation or meta-learning techniques could improve the ranker's generalization and enhance the practical applicability of RankTS by reducing the need for repeated offline training.

  2. Insufficient clarity regarding baseline comparisons and evaluation protocols. The library-selection baselines in Table 1 require further clarification. Specifically, how many complete-model configurations are included in the library, and which models and hyperparameter configurations are considered? Moreover, how does the overall computational efficiency of RankTS compare with library-selection approaches, given that the latter do not require the costly offline task-local evolution process? A more comprehensive comparison that accounts for both offline and target-side computational costs would strengthen the experimental evaluation. In addition, several important evaluation details, such as baseline configurations, the definition of retrieval precision, and the historical-mean ranking baseline, are primarily presented in the appendix. Including these essential details in the main text would improve clarity and reproducibility.

Questions for authors
  1. How could RankTS be extended to multivariate time-series forecasting?

  2. Could the authors provide concrete examples or visualizations of the forecasting architectures represented by DAGs? Additionally, what advantages does the DAG-based representation offer over a simpler sequential architecture, particularly in terms of architectural expressiveness and forecasting performance?

  3. What specific tasks are used during the offline task-local evolution stage? Are these tasks constructed from the same five benchmark datasets used in the main experiments, or are additional datasets involved?

5 comments

W1. Cross-dataset transferability

We agree that the current cross-dataset results do not establish reliable generalization to arbitrary new domains. The original experiment transfers a ranker trained on up to 150 historical series from one dataset. Limited source diversity may contribute to weak transfer, but we have not isolated its causal effect.

We additionally trained on multiple source datasets and evaluated on a held-out dataset at H96. The table reports test MSE.

MethodWeatherElectricity
Multi-source transfer (K=1)1.6742 ± 0.39940.2046 ± 0.0064
RankTS Top11.1860.190
RankTS Top3+V1.1820.181
DLinear1.1950.211
PatchTST1.6820.182
iTransformer1.7790.187
TimesNet1.7410.187
TimeMixer1.6950.196
FEDformer1.8360.248
Autoformer2.1140.299
Informer1.1890.598
Transformer1.2080.341
Nonstationary Transformer1.2860.284
LightTS1.2830.233
Historical best1.2080.187
Meta-select1.2080.187
CM3+V1.1830.182
  • Multi-source transfer is competitive with some baselines, but remains below within-dataset RankTS.

  • Ranker transfer and archive transfer must be distinguished. The original cross-dataset experiment still uses target historical archives. Eliminating target-side offline searches requires both source-only ranker training and source-only archive construction; the new results establish this only if both conditions hold.

  • Adaptation remains a promising extension. Pooled historical training, domain-conditioned ranking, or meta-learning could improve transfer, but their effectiveness requires further evaluation.

W2. Baseline comparisons and evaluation protocols

1. Complete-model library

The library contains 13 configurations from 11 families: DLinear, PatchTST, iTransformer, TimesNet, TimeMixer, FEDformer, Autoformer, Informer, Transformer, Nonstationary Transformer, and LightTS.

  • iTransformer contributes three configurations. The tuples (d_model, n_heads, e_layers, d_ff) are (32,4,1,64), (64,4,2,128), and (128,8,2,256).

  • Each other family contributes one configuration. Other configurable deep baselines use (64,4,2,128) where applicable; PatchTST uses patch length 16 and stride 8.

  • Shared protocol. Predictors use the same data splits, sampled-window limits, and optimization budget within each comparison.

2. Library-selection policies

  • Hist selects the configuration with the lowest mean historical utility loss, using within-task median-centered log validation losses.

  • Select predicts these losses using a multi-output random forest trained on historical series descriptors.

  • CM3+V fits Select's top three configurations and selects the target-validation winner.

Hist and Select each fit one target predictor. CM3+V and RankTS Top3+V share a three-candidate target-fitting budget, although this does not equalize runtime or offline cost. We will make the library contents and selection rules explicit in the main text.

W2. Baseline comparisons and evaluation protocols (continued)

3. Offline and target-side costs

RankTS incurs substantial offline acquisition cost. Each historical task evaluates 450 candidates, with the following average cumulative evaluation times.

DatasetCumulative candidate-evaluation time per task (min)
Weather37.94
London40.02
Wind38.19
KDD37.81
Electricity38.88

At 150 historical tasks per dataset, these values correspond to approximately 94.5–100.1 cumulative candidate-evaluation hours per dataset. They are summed evaluation times, rather than end-to-end elapsed times under parallel execution, and exclude ranker training and other pipeline overhead.

New queries reuse the archive and ranker without repeating evolution. The measured online breakdown is as follows.

DatasetBudgetFeatures + recommendation (s)Training + validation (s)Selected-model test inference (s)
WeatherK=10.013119.701.335
WeatherK=30.013358.221.017
ElectricityK=10.04955.080.723
ElectricityK=30.049181.121.032

Library selection also has an offline cost. Evaluating 13 configurations on 150 historical series requires 1,950 fits per dataset, compared with RankTS's nominal 67,500 candidate attempts. These counts do not directly determine runtime because model costs differ.

For N queries sharing an archive, average cost includes offline acquisition and ranker training divided by N, plus feature extraction, recommendation, target fitting, and inference. Thus, the benefit of reuse depends on deployment volume. The current timings do not establish an overall cost advantage over library selection; a complete comparison also requires library-acquisition and selector-training timings under matched execution conditions.

4. Essential evaluation definitions

We will make the following definitions explicit in the main text.

  • Retrieval Precision@K. For each query, relevance means membership among the 100 lowest-test-MSE architectures in the shared finite candidate set. Precision@K is the fraction of the recommended K candidates in this reference set. Test outcomes define retrospective evaluation only; they do not train the ranker or select the deployed predictor.

  • Historical-mean architecture ranking. Architectures are ordered by their mean within-task historical validation percentile. This differs from Hist, which selects complete models using centered log validation losses.

  • Target-side selection. The final predictor is selected by validation MSE within the prescribed shortlist before test evaluation.

Q1. Extension to multivariate forecasting

Two extensions are possible.

  1. Channel-independent forecasting. Apply RankTS separately to each channel and concatenate the forecasts into a multivariate output. Alternatively, aggregate architecture scores across channels to choose a shared architecture, while forecasting each channel separately. This reuses the current pipeline but does not explicitly model cross-channel dependencies.

  2. Joint multivariate forecasting. Preserve both time and variable axes throughout the operators, use descriptors that characterize cross-channel relationships, and produce a multivariate prediction head. Existing variable-attention and graph operators provide relevant building blocks, subject to compatible multivariate interfaces and training implementations.

These are proposed extensions. The reported experiments fit individual univariate series and do not establish multivariate forecasting performance.

Q2. Concrete architectures and the advantages of DAGs

1. An actual recommended architecture

For Wind series T248, seed 0, the second-ranked architecture decomposes the input using a moving average, processes the smooth and residual branches separately, and fuses them before forecasting. (Figure 12 in the paper)

flowchart TD
    X["Input series X"]
    N0["n0: Moving Average<br/>kernel_size = 25"]
    N1["n1: Identity<br/>input merge: X - n0"]
    N2["n2: Temporal Linear<br/>bias = true"]
    N3["n3: Temporal Linear<br/>bias = true"]
    N4["n4: Temporal Mixer<br/>input merge: n2 + n3<br/>expansion = 1"]
    H["Linear Forecasting Head"]
    Y["Forecast"]
    X --> N0
    X --> N1
    N0 --> N1
    N1 --> N2
    N0 --> N3
    N2 --> N4
    N3 --> N4
    N4 --> H
    H --> Y

We will add concrete architecture visualizations, including operator settings, connections, merges, and prediction heads, together with a chain example illustrating that simple architectures remain valid recommendations.

2. Architectural expressiveness

A sequential architecture is a special case of a DAG. DAGs additionally support branches, skip connections, and intermediate-result reuse, enabling separate processing and fusion of complementary temporal information.

CapabilityStrict sequential architectureDAG architecture
Sequential processingSupportedSupported
Parallel branches on the same inputNot directly in a strict chainSupported
Separate trend/residual processing and fusionRequires branchesDirectly represented
Skip connections and reuse of intermediate resultsNot supported by a strict chainSupported

3. Forecasting performance

We compared error distributions for sequential and branched architectures. The table reports mean MSE and its 25th and 75th percentiles.

DatasetArchitectureMeanQ25Q75
WeatherSequential1.34611.19101.3864
WeatherBranched DAG8.70621.18551.2284
LondonSequential1.22131.17021.2366
LondonBranched DAG11.19441.16931.2176
WindSequential2.75900.37290.5665
WindBranched DAG1.35790.35970.4104
KDDSequential0.54590.49060.5878
KDDBranched DAG1.46740.48290.5640
ElectricitySequential0.28680.20650.2599
ElectricityBranched DAG1.44720.20640.2355
  • Branched DAGs have lower Q25 and Q75 on all five datasets, showing that competitive branched architectures are available.

  • Mean MSE is higher on four datasets, with Wind the exception. This indicates sensitivity to high-error candidates and prevents a blanket claim that branching improves performance.

  • This is a descriptive comparison, not an isolated branching ablation. Establishing a causal forecasting advantage requires matching operator choices, capacity, and evaluation budgets.

Q3. Tasks used for offline task-local evolution

The main offline stage uses historical series from the same five benchmark datasets; no additional datasets are involved.

  • Task definition. Each task is one historical univariate forecasting series. In the main H96 acquisition setup, input length and forecasting horizon are both 96.

  • Offline acquisition. For each dataset, 150 historical series are used. Candidates are fitted on each series' training segment and evaluated on its validation segment; test losses do not guide evolution.

  • Online evaluation. A disjoint pool of 17 series per dataset is reserved for queries. The primary evaluated cohort retains 16 Wind queries and 17 for each other dataset.

The main protocol builds the archive and ranker separately for each dataset. The additional multi-source transfer experiment is a separate evaluation using pooled source datasets and a held-out target dataset.

Reviewer_2aeTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Time series forecasting models perform inconsistently across heterogeneous series, and existing approaches either select from predefined model libraries with limited structural coverage or rely on costly pretrained foundation models. The paper proposes RankTS, a component-level model selection framework that builds a compositional DAG space from 54 forecasting operators. RankTS collects architecture evaluations on historical tasks, and trains a series-conditioned ranker so that only the top-3 candidates need to be fitted on an unseen series. Experiments on five Monash datasets and four horizons compare RankTS with 11 forecasting model families and library-selection baselines, and the authors provided cost analysis, ablations, cross-dataset transfer, and training-budget sensitivity studies. Overall, the idea of selecting and composing models at the operator level is interesting, but concerns remain regarding the construction of the meta-learning framework and its robustness.

Critical strengths
  1. The idea of composing forecasting architectures at the operator level is interesting.

  2. The experiments‘ perspecitve are diverse. Besides the main comparison, the authors study the method from the perspectives of computational budget, ablation, cross-dataset transfer, and training sensitivity.

Critical weaknesses
  1. The related work is insufficient. The work is closely related to neural architecture search, model selection, and transferability estimation, but these lines of research are not adequately discussed. The authors should cover the relevant literature and clarify how RankTS differs from these methods. Here are some references I think is highly related: [1] adopted a series-level performance estimation from time series characteristics and model characteristics [2] discussed the module-level performacne benefit.

[1] Yao, Q., Jin, M., Zhang, C., Yang, C. H. H., Qi, J., & Pan, S. (2025). Estimating Time Series Foundation Model Transferability via In-Context Learning. arXiv preprint arXiv:2509.23695.

[2] Zhao, Zhiyuan, et al. "Timerecipe: A time-series forecasting recipe via benchmarking module level effectiveness." International Conference on Learning Representations. Vol. 2026. 2026.

  1. I am concerned that such a optimization framework may be less robust than time series foundation models. The framework depends on task-pair construction, many manually selected operators and time series characteristics. Even if it outperforms some foundation models, it is much harder to use than zero-shot forecasting. The transparent pipeline of this framework is an advantage, therefore some Insights into what components suit what kinds of series could therefore be more valuable than the performance gains alone.

  2. Many components of the framework rely on elaborate design choices, such as the operator library, the time series characteristics, and the number of task pairs. Simplifying the pipeline and reducing its sensitivity to hyperparameters and human priors would make the framework more valuable.

  3. In the experiments, all predictors, including deep models such as PatchTST and iTransformer, are trained for at most five epochs, which likely leaves them undertrained. Training with early stopping until convergence would give a fairer comparison. Moreover, the introduction argues that time-series foundation models have clear disadvantages, so a comparison with several such models would help support this claim.

Questions for authors
  1. The online stage only selects architectures from a fixed archive built on historical series of the same dataset. How would RankTS ensure its generalization to a new domain without related historical searches?

  2. Factors such as forecasting length, context length, and the prediction mode (one-step, multi-step or autoregressive) may also affect which architecture works best. Should architecture selection take these factors into account?

6 comments

W1. Related work and positioning

We thank the reviewer for these relevant references. We will expand the related work to clarify the connections to neural architecture search (NAS), model selection, and transferability estimation.

  • NAS. NAS approaches such as TimeRecipe search for architectural configurations within a predefined design framework, for example by optimizing feature extraction modules. RankTS focuses on evolving new model architectures from basic components, rather than refining configurations within an established architectural template.

  • Model selection. Model selection typically involves choosing among a given set of existing models. RankTS extends beyond this setting by evolving new architectures from basic components, thereby generating additional candidates for selection.

  • Transferability estimation. TimeTic predicts the post-fine-tuning performance of existing foundation models to guide model selection. Its focus is therefore on estimating the suitability of existing models, whereas RankTS also constructs new model architectures.

These distinctions concern the scope of the compared approaches, rather than a strict separation between research areas: constructing architectures from basic components also relates to the broader NAS literature.

W2. Robustness, usability, and component insights

Zero-shot forecasting is simpler to deploy. RankTS requires offline acquisition and target-side fitting, so forecasting accuracy alone does not establish greater robustness or usability.

Foundation-model comparison. We evaluated Chronos-2, TimesFM 2.5 200M, and Moirai 2.0-R-small without target-specific fine-tuning. The table reports test MSE; Top1 and Top3+V denote RankTS.

DatasetHQueriesTop1Top3+VChronos-2TimesFM 2.5Moirai 2.0
Weather96171.1861.1822.04012.19841.8763
Weather192171.0501.0353.09842.42702.1267
Weather336170.9930.9813.39032.13202.5337
Weather720171.3021.2934.01402.32204.8048
London96171.1741.1841.43351.41031.2938
London192171.2001.2051.36801.48381.3416
London336171.2091.2041.39411.51131.3643
London720171.2601.2501.41801.58151.4397
Wind96160.3370.3290.45770.44530.4206
Wind192160.5030.5020.76300.71760.6634
Wind336160.6340.6321.07671.00310.9188
Wind720160.8340.8351.62591.42121.2908
KDD96170.4910.4800.63980.62340.6386
KDD192170.5030.4890.68710.66040.6834
KDD336170.5070.4980.72230.68890.7492
KDD720170.5010.4960.69900.69590.7884
Electricity96170.1900.1810.21570.22780.1979
Electricity192170.2010.1920.22090.21730.1965
Electricity336170.2220.2100.26790.23770.2171
Electricity720170.2700.2470.34800.28340.2638

Both RankTS variants achieve lower MSE in all 20 settings. This supports competitiveness under this protocol, rather than superiority over fine-tuned foundation models or robustness to arbitrary domain shifts.

Component insights. We analyzed operator prevalence in predicted Top10 architectures and the best 10% by measured validation loss. Enrichment statistics are reported below.

Predicted Top10

OperatorModel presenceNode shareArchive presenceEnrichmentSize-matched enrichment
temporal_mixer56.1%12.2%16.8%3.1833.295
identity53.3%16.1%62.6%0.8420.908
moving_average26.4%5.4%37.2%0.7310.785
temporal_linear22.4%7.2%30.9%0.7730.809
dropout22.1%4.9%18.8%1.0991.085
gelu12.8%2.7%9.4%1.2401.213
batch_norm12.7%2.8%11.2%1.1421.161
variable_attention11.6%2.4%9.5%1.1581.016
relu9.7%2.0%7.1%1.2391.222
segment_recurrent9.3%1.8%9.5%0.6890.670
pyramidal_attention9.2%2.0%6.2%1.5691.507
channel_linear9.2%2.1%9.5%0.9510.810

Best 10% by validation loss

OperatorGood-model presenceArchive presenceSize-matched enrichment
temporal_mixer33.9%16.8%1.940
temporal_linear41.1%30.9%1.356
gelu11.9%9.4%1.249
spectral_filter6.9%5.9%1.084
variable_attention9.9%9.5%1.058
temporal_attention4.8%4.7%1.037
moving_average38.9%37.2%1.027
identity65.2%62.6%1.023
gru8.2%7.9%1.011
two_stage_attention7.2%6.8%1.005
  • Temporal mixing shows the clearest enrichment, with size-matched enrichment of 3.295 in predicted Top10 and 1.940 in the validation-selected group.

  • Temporal linear and GELU are also enriched in the validation-selected group (1.356 and 1.249).

  • These are associations within the evaluated archive, not causal component effects or rules linking specific series properties to optimal operators. Such rules require a feature-stratified analysis.

W3. Design complexity and sensitivity

We agree that the current pipeline retains human priors and consequential design choices. The existing ablations help identify their roles.

  • Series descriptors. catch22 performs better than the tested window-based CNN for recommendation, supporting a compact statistical representation.

  • Pair construction. Balanced sampling improves Precision@3 from 34.1% to 47.0% relative to uniform sampling, demonstrating that this choice matters.

  • Training objective. The auxiliary gap loss provides no mean head-retrieval improvement, supporting the simpler pairwise objective.

These results do not establish insensitivity to operator-library design or pair count. We will distinguish supported choices from remaining sensitivities. Further simplification and automated configuration remain future work.

W4. Training budget and foundation-model baselines

1. Five epochs do not establish convergence

We agree that a shared five-epoch cap does not guarantee equally converged models. The main results measure performance under a limited training budget. We additionally compared five- and twenty-epoch runs at H96, seed 0.

Below, W = Weather, E = Electricity; values are test MSE. Reduction is the reported relative MSE decrease from five to twenty epochs; negative values indicate an increase. A dash denotes an unreported result.

MethodW:5W:20WE:5E:20E
RankTS Top11.19121.18840.23%0.20120.177511.79%
RankTS Top3+V1.18941.18510.36%0.18120.16926.66%
DLinear1.19541.19490.04%0.21000.19556.90%
PatchTST1.69511.67910.94%0.18170.16857.28%
iTransformer1.78051.7989-1.04%0.18670.16909.51%
TimesNet1.73321.7540-1.20%0.18150.17135.60%
TimeMixer1.70521.69500.60%0.19440.169712.66%
FEDformer2.0773——0.2496——
Autoformer2.04852.00991.88%0.30050.27109.80%
Informer1.17881.17370.43%0.60480.277554.12%
Transformer1.18781.18470.26%0.34440.208039.60%
Nonstationary Transformer1.27511.2857-0.83%0.29140.208628.40%
LightTS1.29611.25922.85%0.23390.188819.26%
Hist1.18781.18470.26%0.18150.17135.60%
Select1.18781.18470.26%0.18180.17125.84%
CM3+V1.19531.18730.67%0.17890.16557.49%

Longer training can materially improve baselines, especially on Electricity. For example, Informer improves from 0.6048 to 0.2775. These results therefore do not justify assuming that five epochs suffice for every model.

2. Additional twenty-epoch results

MethodWeather H96Electricity H96
RankTS Top11.1717 ± 0.01460.1727 ± 0.0051
RankTS Top3+V1.1688 ± 0.01420.1688 ± 0.0019
DLinear1.1948 ± 0.00240.1962 ± 0.0011
PatchTST1.6770 ± 0.01310.1690 ± 0.0009
iTransformer1.7872 ± 0.04580.1684 ± 0.0005
TimesNet1.8086 ± 0.05230.1741 ± 0.0028
TimeMixer1.6874 ± 0.01400.1710 ± 0.0012
FEDformer1.7017 ± 0.39240.2317 ± 0.0069
Autoformer2.0263 ± 0.02420.2776 ± 0.0063
Informer1.1843 ± 0.01000.2779 ± 0.0071
Transformer1.1918 ± 0.02990.2145 ± 0.0062
Nonstationary Transformer1.2864 ± 0.01920.2071 ± 0.0014
LightTS1.2393 ± 0.02080.1876 ± 0.0013
Hist1.1918 ± 0.02990.1741 ± 0.0028
Select1.1918 ± 0.02990.1741 ± 0.0028
CM3+V1.1948 ± 0.03060.1672 ± 0.0018
  • Weather H96. RankTS Top3+V has the lowest reported mean MSE (1.1688).

  • Electricity H96. RankTS Top3+V remains competitive (0.1688), while CM3+V (0.1672) and iTransformer (0.1684) have lower means.

We therefore frame the evidence as competitiveness under extended training, rather than universal superiority. A twenty-epoch cap still does not establish convergence; training with validation-based early stopping until convergence remains a stronger comparison.

3. Foundation-model comparison

The complete comparison with Chronos-2, TimesFM 2.5, and Moirai 2.0 is reported in our response to W2. We will replace blanket statements about foundation-model disadvantages with a narrower discussion of task-specific accuracy, adaptation cost, and deployment requirements.

Q1. Generalization to a domain without related historical searches

Within-dataset generalization and unseen-domain generalization are different settings. The main 150-history/17-query split tests unseen series within a dataset; it does not establish transfer to an unrelated domain.

RankTS can apply a source-trained ranker to new series descriptors, but success depends on source coverage and archive suitability. The paper's cross-dataset ranker experiment still uses target historical archives, so it does not demonstrate deployment without target historical searches.

We additionally trained on multiple source datasets and evaluated on a held-out dataset at H96.

MethodWeatherElectricity
Multi-source transfer (K=1)1.6742 ± 0.39940.2046 ± 0.0064
RankTS Top11.1860.190
RankTS Top3+V1.1820.181
DLinear1.1950.211
PatchTST1.6820.182
iTransformer1.7790.187
TimesNet1.7410.187
TimeMixer1.6950.196
FEDformer1.8360.248
Autoformer2.1140.299
Informer1.1890.598
Transformer1.2080.341
Nonstationary Transformer1.2860.284
LightTS1.2830.233
Historical best1.2080.187
Meta-select1.2080.187
CM3+V1.1830.182

Multi-source transfer is competitive with some baselines but remains below within-dataset RankTS. Its Weather MSE is 1.6742, compared with 1.186 for within-dataset Top1; Electricity MSE is 0.2046 versus 0.190.

These results are preliminary evidence for transfer, not a guarantee. Demonstrating deployment without target historical searches additionally requires source-only archive construction, as well as source-only ranker training. The reported transfer results do not settle this requirement unless both conditions hold.

Q2. Forecast horizon, context length, and prediction mode

Yes. These factors can affect architecture suitability and should be considered when extending RankTS. The current framework fixes context length and prediction protocol, and reuses H96 rankings across forecasting horizons while fitting predictor weights separately.

To examine horizon sensitivity, we compared architecture rankings at forecasting lengths 64 and 128.

Datasets with five complete queries

DatasetRankingSpearman rho95% CIKendall tau-bTop10 overlap
Electricityvalidation0.9526[0.9283, 0.9682]0.838474%
Electricitytest0.9504[0.9402, 0.9606]0.827168%
KDDvalidation0.9121[0.8507, 0.9637]0.769658%
KDDtest0.8790[0.8172, 0.9407]0.728866%
Londonvalidation0.9565[0.9342, 0.9792]0.848284%
Londontest0.7923[0.6545, 0.9301]0.654254%

Datasets with four complete queries

DatasetRankingSpearman rhoKendall tau-bTop10 overlap
Weathervalidation0.89490.730257.5%
Weathertest0.85040.681040.0%
Windvalidation0.74790.579020.0%
Windtest0.77470.601342.5%
  • Global rankings are substantially correlated, with Spearman rho ranging from 0.7479 to 0.9565.

  • Top candidates can still change. Top10 overlap is only 20% for Wind validation rankings and 40% for Weather test rankings. Global correlation does not ensure identical best architectures.

  • The evidence concerns horizon changes only. It does not establish invariance to context length or direct versus autoregressive prediction.

These results support ranking reuse as a practical approximation in the tested settings. A broader framework could condition the ranker on horizon, context length, and prediction mode, provided historical evaluations cover those configurations.

Reviewer_62kmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposed a method called RankTS, that first creates a pool of ~2k neural networks model candidates by combining various components , then trains on these neural networks architectures with time series, and in the end uses the trained framework to select the best model(s) for forecasting during inference. RankTS has some novelly proposed steps, including a coverage and diversity aware initialization, task-local evoluation, and a graph encoder structure.

Critical strengths
  1. The idea of time-series forecasting model selection based on compoenent-level is novel and inspiring.
Critical weaknesses
  1. Handcrafting model candidates in RankTS requries labor-intensive human examination. The process is tedious and the resulting pool doesn't generalize well if more components are introduced.
  2. The proposed method is limited in selecting models in the candidate pool, which only includes ~2k models constructed by basic components. The comparison against other baselines is then difficult because other baselines can rely on a pool of advanced candidates. For example, baselines can include Chronos and other foundation models in their candidate pool, and recent research has shown that time series foundation models generally outperform models like those in the candidate.
  3. Another limit is the model candidates only have 10 components at most.
  • The fontsize is too small to read in the Figure 2.
Questions for authors
  1. What should be considered as a component? Can commonly used NN structures such as transformer and cross-attention transformation be considered structures, and also GNN or RNN? Does Table 7 include all?
  2. Are the 11 operator families such as Linear, Local, etc. in line 141?
  3. Can the model generation process (from the base architectures) evolve to complicated models such as transformer.
  4. How should @K=? be practically determinied? Should K=3 be the default for all datasets?
  5. In line 360-362, ideally, the larger budget only enables the framework to use more models, but it shouldn't enforce the framework in using low-ranked candidates. Can this be fixed in RankTS?
7 comments

Responses to Critical Weaknesses (W1–W2)

We thank the reviewer for the constructive feedback.

W1. Human effort and extensibility

Candidate generation and evolution are automated once the operator library and generation rules are specified. Human input mainly involves two tasks.

  • Operator specification. Identifying reusable operators and defining their interfaces and hyperparameters.

  • Rule specification. Defining base recipes and checks for compatibility, validity, and redundancy.

Individual generated architectures do not require manual inspection. The 282 base architectures are expanded into 1,965 valid initial candidates, followed by task-local evolution that generates additional architectures.

RankTS provides a framework for constructing and recommending architectures. Coverage- and diversity-aware initialization promotes representation of configured operators, while evolution explores their combinations using historical performance. Adding operators requires specifying their interfaces and compatibility rules, and potentially rebuilding the archive and ranker. Performance under arbitrary library expansions has not yet been established.

W2. Candidate-pool size and foundation-model comparisons

The initial pool and deployment archive serve different purposes.

  • Initial pool. The 1,965 valid candidates provide starting architectures for offline exploration.

  • Evolution. Task-local evolution generates additional candidates beyond this pool.

  • Deployment archive. Online recommendation selects from an archive of up to 2,000 architectures constructed using historical observations.

We additionally evaluated Chronos-2, TimesFM 2.5 200M, and Moirai 2.0-R-small without target-specific fine-tuning. The table reports test MSE.

DatasetHorizonQueriesRankTS Top1RankTS Top3+VChronos-2TimesFM 2.5Moirai 2.0
Weather96171.1861.1822.04012.19841.8763
Weather192171.0501.0353.09842.42702.1267
Weather336170.9930.9813.39032.13202.5337
Weather720171.3021.2934.01402.32204.8048
London96171.1741.1841.43351.41031.2938
London192171.2001.2051.36801.48381.3416
London336171.2091.2041.39411.51131.3643
London720171.2601.2501.41801.58151.4397
Wind96160.3370.3290.45770.44530.4206
Wind192160.5030.5020.76300.71760.6634
Wind336160.6340.6321.07671.00310.9188
Wind720160.8340.8351.62591.42121.2908
KDD96170.4910.4800.63980.62340.6386
KDD192170.5030.4890.68710.66040.6834
KDD336170.5070.4980.72230.68890.7492
KDD720170.5010.4960.69900.69590.7884
Electricity96170.1900.1810.21570.22780.1979
Electricity192170.2010.1920.22090.21730.1965
Electricity336170.2220.2100.26790.23770.2171
Electricity720170.2700.2470.34800.28340.2638

Both RankTS variants achieve lower MSE across all 20 settings, supporting task-specific architecture recommendation under this protocol. These results do not establish superiority over fine-tuned foundation models or selection methods using a foundation-model-enriched pool.

Responses to W3–W4 and Q1–Q3

W3. The ten-component limit

The ten-node cap is a practical restriction of the current implementation, rather than an intrinsic requirement of RankTS. A node can represent an attention, convolutional, or recurrent module, so node count does not directly correspond to network depth or parameter count.

We analyzed architectures containing one to ten nodes. Entries report mean MSE [25th percentile, 75th percentile].

NodesWeatherLondonWindKDDElectricity
11.2119 [1.1837, 1.2135]1.2018 [1.1677, 1.2266]1.9445 [0.3745, 0.4538]0.5394 [0.4886, 0.5235]0.2224 [0.2109, 0.2343]
21.2852 [1.1877, 1.2728]1.2058 [1.1661, 1.2224]3.8286 [0.3596, 0.4522]0.5249 [0.4819, 0.5651]0.2634 [0.2096, 0.2387]
31.2687 [1.1872, 1.2483]1.2102 [1.1693, 1.2284]0.5337 [0.3607, 0.4398]0.5263 [0.4845, 0.5574]0.2330 [0.2057, 0.2380]
42.2683 [1.1835, 1.2398]2.1384 [1.1656, 1.2195]0.8924 [0.3567, 0.4149]0.7524 [0.4847, 0.5653]0.3248 [0.2031, 0.2352]
510.2601 [1.1867, 1.2330]12.8341 [1.1704, 1.2210]1.8174 [0.3616, 0.4267]1.3703 [0.4819, 0.5625]1.8570 [0.2089, 0.2362]
67.3697 [1.1862, 1.2313]9.3173 [1.1704, 1.2212]2.0135 [0.3664, 0.4351]1.3871 [0.4841, 0.5675]1.5570 [0.2066, 0.2383]
71.4278 [1.1970, 1.7098]1.2226 [1.1727, 1.2328]2.8946 [0.3746, 0.5597]0.5446 [0.4952, 0.6020]0.2594 [0.2044, 0.2341]
81.5102 [1.2319, 1.7099]1.2421 [1.1746, 1.2576]5.0928 [0.3825, 0.6460]0.5552 [0.5065, 0.6090]0.4196 [0.2555, 0.4989]
91.7361 [1.7094, 1.7101]1.4299 [1.4161, 1.4437]0.6140 [0.4062, 0.5499]0.5617 [0.5172, 0.6106]0.6533 [0.5209, 0.8433]
101.9928 [1.7096, 2.1347]2.0045 [1.6063, 2.4027]0.4743 [0.3931, 0.5162]0.5459 [0.4944, 0.5335]1.5103 [1.5103, 1.5103]
  • More nodes do not consistently improve performance. Larger architectures exhibit worse error distributions on several datasets.

  • The evidence supports compact architectures within the evaluated space. Potential benefits from architectures exceeding ten nodes remain open, since they were not evaluated.

W4. Figure 2 readability

Thank you for pointing this out. We will enlarge the text and simplify the layout of Figure 2 to improve readability at publication size.

Q1. Component definition and coverage

A component is a configurable computational operator represented by one DAG node. Its granularity follows the library interface; it need not be mathematically indivisible.

  • Included operators. The library contains temporal, variable, patch, two-stage, and pyramidal attention; GRU, LSTM, and segment recurrence; and graph operators such as adaptive graph convolution and dynamic graph filtering.

  • Transformer. A complete Transformer combines multiple building blocks. Reproducing it also requires suitable connectivity and node capacity.

  • Cross-attention. Explicit cross-attention is not separately listed and would require appropriate query and key/value interfaces.

Table 6 lists all 54 operators and their families. Table 7 specifies their discrete hyperparameters, covering 227 operator–hyperparameter configurations.

Q2. The eleven operator families

The families are Linear, Local, Attention, Recurrence, Trend, Frequency, Mixing, Dynamics, Variable graphs, Support, and Activation/position. They contain 54 operators in total, as listed in Appendix C.1, Table 6.

Q3. Evolution toward complex architectures

Evolution can generate architectures beyond the base recipes through operator replacement, hyperparameter changes, node addition/deletion, and modifications to skip inputs and residual connections.

Transformer-like compositions are possible in principle when the operators, connectivity rules, and node budget permit them. However, reliable generation of complete Transformers has not been demonstrated. The current study focuses on compact task-specific architectures, for which greater complexity does not consistently improve performance, as shown in W3.

Responses to Q4–Q5

Q4. Practical choice of K

K=3 is a practical default, not a universally optimal choice. K determines how many recommended candidates are trained and validated on the target series.

The table illustrates the accuracy–cost trade-off. Training and validation times are cumulative across evaluated candidates.

DatasetKTest MSE ↓Test MAE ↓Training time (s)Validation time (s)Selection time (ms)
Weather11.19120.647524.191.350.002
Weather31.18940.6465154.849.260.002
Weather51.19250.6491324.7019.570.002
Weather101.19220.6492835.8149.880.003
Weather201.18800.64461854.91113.150.005
Weather501.18240.64254090.62250.920.013
Weather1001.18220.64248049.87492.110.023
Electricity10.20120.312150.502.770.002
Electricity30.18120.290097.415.500.001
Electricity50.18070.2894148.578.500.002
Electricity100.17970.2894353.6520.670.002
Electricity200.18020.2898718.9442.610.004
Electricity500.17830.28751866.41114.410.010
Electricity1000.17830.28724388.84269.440.019

Larger K can provide additional gains, but test error is not monotonic and fitting costs increase substantially. For example, increasing K from 3 to 100 reduces Electricity MSE from 0.1812 to 0.1783 while increasing training time from 97.41 to 4388.84 seconds.

Our practical recommendations are as follows.

  • Strict fitting budget. Use K=1.

  • Compact default. Start with K=3 to allow target-specific validation among several recommendations.

  • Additional budget. Increase K when representative historical validation results justify the expected benefit.

The choice should follow the fitting budget and validation evidence. Test data should not be used to tune K.

Q5. Does a larger budget force the use of lower-ranked candidates?

No. A larger K expands the shortlist, but RankTS deploys only the eligible candidate with the lowest validation MSE. Three quantities should be distinguished.

  1. Shortlist precision. Lines 360–362 concern Precision@K, the fraction of recommendations belonging to the reference set of the 100 best candidates. This fraction can decrease as the shortlist expands.

  2. Selected validation error. For a fixed ranking and fixed fitted-candidate results, larger prefixes retain all previous candidates. The minimum validation MSE therefore cannot worsen.

  3. Selected test error. Test MSE can still increase because a candidate’s validation advantage does not always generalize to testing.

The existing selection step therefore preserves the option to retain a candidate from the smaller shortlist. Treating K as a maximum budget and adding validation-based early stopping is a possible extension, but would require separate evaluation.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for their constructive feedback. We have made targeted revisions to clarify RankTS, strengthen the evaluation, and make its limitations explicit. The table maps these changes to the weakness (W) and question (Q) numbering within each review.

Key comparisons and clarifications appear in the main text. Detailed protocols and supplementary results are provided in the appendices, with pointers from the main text. The main text remains within ten pages.

ReviewerCommentsChangesLocation
62kmW1Clarified automated candidate generation, validity checks, and evolution, with human-defined operator interfaces and rules. Explained the requirements and remaining limitations of library expansion.Secs. 3.1.1–3.1.3; Algorithms 1–2; Apps. C.1, D; Sec. 5
62kmW2Distinguished the initial pool from the deployment archive capped at 2,000 architectures. Added zero-shot Chronos-2, TimesFM 2.5, and Moirai 2.0 comparisons.Secs. 3.1.2–3.1.3, 4.1; Table 1; Apps. F.6, G.6
62kmW3Added error quartiles for node counts 1–10. Clarified that the ten-node cap is practical and larger architectures remain unevaluated.Sec. 4.3; Fig. 5; App. C.4; Table 8
62kmQ1–Q3Defined components and documented 54 operators, 11 families, and 227 configurations. Clarified attention/RNN/GNN support, cross-attention interfaces, and limits of generating complete Transformers.Sec. 3.1.1; App. C.1; Tables 6–7
62kmQ4–Q5Explained K=3 as a practical default and added budgets up to K=100. Clarified validation selection and why larger prefixes need not improve selected test error.Secs. 3.2, 4.2; Fig. 3; Apps. G.2, G.5; Table 22
2aeTW1Revised Related Work and incorporated TimeTic, TimeRecipe, AutoCTS, and evolutionary NAS, clarifying their relationship to RankTS.Sec. 2
2aeTW2–W3Presented representation, scorer-input, pair-construction, and objective ablations. Added component composition and size-matched enrichment analyses. Acknowledged untested library-expansion and pair-count sensitivity.Secs. 4.3–4.4; Figs. 4, 8; Apps. E.3–E.4, I.5, K.1–K.3; Sec. 5
2aeTW4Added foundation-model comparisons and paired five-/20-epoch-cap experiments with validation early stopping on Weather and Electricity at H96/H192. Retained the original five-/30-epoch shortlist analysis.Secs. 4.1, 4.6; Tables 1, 3; Apps. F.6–F.7, G.4; Tables 20–21
2aeTQ1Added ranker transfer from four source datasets to each held-out target. Explicitly stated that target archives remain necessary.Sec. 4.5; App. J.4; Table 34; Sec. 5
2aeTQ2Added H64/H192 ranking-sensitivity results. Clarified that context length and prediction mode remain fixed.App. K.7; Tables 44–45; Sec. 5
gJXyW1Added multi-source ranker transfer and distinguished transferred scoring from architecture acquisition without target history.Sec. 4.5; Fig. 9; App. J.4; Table 34; Sec. 5
gJXyW2Specified 13 configurations from 11 model families and CM3+V. Clarified Precision@K and historical-mean ranking. Added offline/target-side costs, amortization, and qualified cost-crossing estimates.Sec. 4 setup; Sec. 4.2; Table 1; Fig. 3; Apps. F.1–F.3, F.8, J.1–J.2; Tables 11–15
gJXyQ1Discussed channel-independent and joint multivariate extensions, required interfaces, and their unevaluated status.App. I.6; Sec. 5
gJXyQ2Added a concrete executable DAG, topology comparisons, node-count matching, and residual sensitivity. Qualified these analyses as observational.Apps. I.3–I.4; Fig. 12; Tables 26–28
gJXyQ3Clarified that offline tasks use the same five benchmark datasets, with 150 historical series per dataset and disjoint query series.Sec. 4 setup; Apps. B.1, D

The revised paper explicitly retains the remaining evidence boundaries. Extended training does not certify convergence for every predictor; transfer remains conditional on target archives; and component/topology associations do not establish causal benefits.

AuthorsOpenReview ↗

Summary of the Discussion

Dear Reviewers, Area Chairs, Senior Area Chairs, and Program Chairs,

We sincerely thank you for your time and constructive feedback. We have incorporated the reviewers' suggestions into additional experiments and targeted manuscript revisions. Below, we summarize RankTS's contributions, the strengthened empirical evidence, and reviewer feedback during the discussion.

Contributions and Reviewer Recognition

  • Beyond fixed model libraries. RankTS automatically builds forecasting models from reusable components, enabling series-specific architecture construction and selection. 62km recognized this perspective as “novel and inspiring.”

  • Learning model preferences through ranking. RankTS learns series-specific architectural preferences from historical evaluations and jointly models time series and architectures to recommend promising models for unseen series.

  • Strong results with few target-side fits. Using only three candidate fits per target, Top3+V ranks first in 11/20 settings and among the top two in 16/20, achieving lower MSE than all three evaluated zero-shot foundation models in all 20 settings.

Responses to Main Concerns and Additional Experiments

We incorporated the following additional experiments into the revised manuscript.

  • Foundation-model comparisons (62km; 2aeT). Top3+V achieves lower MSE than zero-shot Chronos-2, TimesFM 2.5, and Moirai 2.0 in all 20 settings. (Table 1; Appendix F.6.)

  • Architecture-size analysis (62km). Analyses of 1–10-node architectures support compact designs and clarify the practical ten-node budget. (Figure 5; Appendix C.4.)

  • Recommendation-budget analysis (62km). Accuracy–cost comparisons for K = 1–100 support K = 3 as an effective, economical default. (Appendix G.5; Table 22.)

  • Component-preference insights (2aeT). Size-matched operator analyses reveal temporal-mixing enrichment in both recommended and validation-best architectures. (Figure 8; Appendix I.5.)

  • Extended-training comparisons (2aeT). Paired 5-/20-epoch-cap experiments with validation early stopping on Weather and Electricity (H96/H192) show Top3+V achieves the lowest mean MSE in three of four settings. (Appendix F.7; Tables 20–21.)

  • Multi-source ranker transfer (2aeT; gJXy). We trained rankers on 600 tasks from four source datasets and evaluated transfer on held-out Weather or Electricity using target archives. (Appendix J.4; Table 34.)

Discussion Outcome

The paper received initial ratings of 3, 2, and 2. The official ratings remain unchanged as of November 19, 2026, with neither of the two reviewers who initially assigned 2 having recorded a rating update following the rebuttal.

  • gJXy (initial rating 3) confirmed that “Most of my questions have been adequately addressed” and reaffirmed support for acceptance.
  • 62km (initial rating 2) thanked us for addressing their comments and recognized the paper as “valuable in providing insight on the topic.” We subsequently provided a direct link to the revised manuscript following their reported access difficulty.
  • 2aeT (initial rating 2) had not provided a follow-up response to our detailed rebuttal and additional experiments at the time of this summary, despite our follow-up messages.

We respectfully ask the Area Chair to consider the revised manuscript, additional experimental evidence, and positive discussion feedback alongside the initial numerical ratings when making the final assessment.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.