acceptodds
Under review as a conference paper at ICLR 2027

Can Time Series Foundation Models Scale at Test Time?

Abstract

Test-time scaling aims to improve forecasts through additional inference computation, yet its benefits beyond strong raw-input baselines remain unclear. We study four frozen time series foundation models (TSFMs) on univariate point-forecasting tasks from eight datasets, distinguishing fixed-input sampling, multi-view inference, and history-based selection and aggregation. Repeated sampling improves forecasts with diminishing returns: on eight designated series, doubling the sample count from 64 to 128 reduces average relative MAE by less than 0.6% for each stochastic model. Multi-view aggregation can provide further gains. Across 48 additional series from three datasets, Moirai's historically selected multi-view ensemble achieves a dataset-balanced mean MAE reduction of 7.42% against a historically selected raw-input ensemble. However, gains vary across models and tasks, and stronger raw-input controls reduce some apparent improvements. Historical improvements can reverse at test time, while candidate-pool expansion can worsen selection and incur additional calibration costs. Our findings distinguish gains from sampling, views, and aggregation, highlighting the gap between candidate potential and forecasting improvements realizable using observed history. Code is available at [https://anonymous.4open.science/r/forecasting-tts-fb18b6a3/](https://anonymous.4open.science/r/forecasting-tts-fb18b6a3/).

Primary area:
learning on time series and dynamical systems
Keywords:
Time-Series Foundation Models, Test-Time Scaling, Time-Series Forecasting

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SHJ7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents an empirical study of test-time scaling for frozen time series foundation models. It investigates three sources of potential improvements: repeated sampling, multi-view inference through input transformations, and history-based selection and aggregation. Experiments on four TSFMs across eight datasets show diminishing returns from repeated sampling and model-dependent benefits from multi-view aggregation, while highlighting the importance of strong raw-input baselines, historical selection reliability, and inference costs.

Critical strengths

Well-designed controls that help identify the sources of performance improvements.

The paper goes beyond comparing multi-view inference with a single raw-input baseline. By introducing historically selected raw configurations, raw-input ensembles, and matched residual-predictor controls, it provides a more rigorous assessment of the incremental benefits of input transformations and aggregation, and is valuable for identifying potentially misleading improvements in prior or future studies.

Comprehensive empirical analysis with informative negative findings.

The paper examines multiple aspects of test-time inference, including sampling saturation, input transformations, aggregation strategies, historical selection, candidate-pool expansion, and computational costs. It reports both positive and negative results rather than focusing exclusively on favorable comparisons. In particular, the observations that historical improvements may not transfer to test data and that expanding the candidate pool can degrade performance provide useful practical insights into the limitations of test-time inference strategies.

Critical weaknesses

Limited novelty and insufficient new insights beyond established principles.

Although systematically evaluating these phenomena in TSFMs has empirical value, the paper does not establish sufficiently distinctive insights into why or when test-time scaling is effective. For example, it does not identify model or data characteristics that explain the effectiveness of multi-view inference, nor does it establish generalizable relationships between additional inference computation and forecasting improvements.

Insufficient attribution and limited generalizability of the main positive results.

The strongest positive evidence is concentrated in Moirai, which achieves a 7.42% average MAE reduction over raw top-k ensembles on 48 additional series. The remaining models show considerably smaller or inconsistent improvements. Besides, the relationship between performance improvements and additional test-time computation is not fully characterized. Different sampling budgets lack directly measured runtime comparisons, and a substantial portion of the multi-view overhead comes from historical calibration rather than online prediction.

Questions for authors

Explain the gains from multi-view aggregation while the individual transformed view families generally underperform the raw baseline.

Table 8 shows that none of the three transformed view families improves average MAE over selected raw on Moirai's 48 additional series, yet their joint top-3 ensemble achieves substantial improvements. More analysis is needed to establish whether the gains arise from specific complementary representations or from generic ensembling effects.

5 comments
AuthorsOpenReview ↗

Response to Reviewer SHJ7 (1/3)

Thank you for recognizing the value of our controlled comparisons and for asking what drives the multi-view gains. Our framework changes the temporal representation presented to a frozen TSFM, then uses historical forecasts to select and combine useful candidates. The updated paper connects this design to three findings: cycle views supply better candidates in the Moirai study, compact aggregation improves accuracy across seven models, and historical view selection can retain most of the gain while reducing prediction work.

Explain the gains from multi-view aggregation while the individual transformed view families generally underperform the raw baseline. ... whether the gains arise from specific complementary representations or from generic ensembling effects.

Response: Cycle views supply useful candidates, and historical selection plus compact aggregation improves on raw-input ensembling. On the original 48 additional Moirai series, selecting just one forecast from raw+cycle reduces MAE by 4.52% relative to the historically selected raw top-k Median ensemble. Combining the top three candidates increases this reduction to 7.67%. The gain is already present under single selection and increases with aggregation.

The original family table restricts every series to forecasts from one transformed family. The mixed pool keeps raw forecasts available and lets each series select candidates using its own history. To identify which views contribute, we compare the following pools on those same 48 series, matching historical scoring, Median aggregation, and available ensemble size. Reductions average per-series relative MAE changes with equal dataset weights; positive values indicate improvement.

Candidate poolMAE reduction vs raw top-k Median
Full multi-view7.42%
Raw + cycle7.67%
Full pool without cycle0.42%
Raw + trend0.27%
Raw + subseries0.64%

Removing cycle reduces the gain from 7.42% to 0.42%, while raw+cycle retains the full pool's improvement. Keeping raw available matters: forcing a cycle-only winner increases MAE by 0.88% against the same raw reference. These ablations identify the value of cycle candidates under matched aggregation; pool sizes and calibration work can differ.

Cycle forecasts appear in 46/48 selected sets, and 24/48 sets contain only cycle candidates with different context lengths or periods. The Median ensemble outperforms every selected constituent on 25/48 series. Useful alternatives therefore also arise within a view family.

The Mean-ensemble diagnostic gives a further answer to the attribution question: better selected members dominate the improvement. Relative to the matched raw Mean ensemble, improved member quality contributes 21.3 percentage points, while weaker error cancellation offsets 5.4 points, yielding a net 15.9% MAE reduction after rounding. These numbers concern Mean aggregation; the 7.42% result above uses Median. Sections 4.2–4.3/Figure 2 and Appendix E.3 report these controls.

AuthorsOpenReview ↗

Response to Reviewer SHJ7 (2/3)

... the paper does not establish sufficiently distinctive insights into why or when test-time scaling is effective. ... model or data characteristics that explain the effectiveness of multi-view inference ...

Response: The findings connect the representation supplied to a model with the forecasts worth retaining. Structured views provide alternative forecasts even for direct-quantile models, whose repeated calls on an unchanged input produce no new candidates. On eight Chronos-2 KDD series with two forecast origins, aligned cycle reduces MAE by 9.50% versus raw and 5.26% versus a half-cycle shift, while centering and scale matching have approximately zero effect. This control identifies a benefit from aligned temporal structure in that panel (Figure 2; Appendix E.4).

Compact aggregation makes the gains more consistent across models. With the same single-selection rule applied to multi-view and raw-only pools, aggregate MAE improves in three of seven models; with top-3 applied to both pools, it improves in all seven (Table 1(b)). Within the multi-view pool, top-3 also has lower aggregate MAE than single selection and full-pool averaging in every model, with overall reductions of 1.90% and 3.22%, respectively. The results support generating structured candidates and combining a few that perform well historically.

For a new model and dataset, we use historical trials to determine which views help that model at the intended horizon and loss. Table 4 provides a concrete example: on the same Solar source, eight pilot series select raw+cycle for Chronos-2, raw+trend for TimesFM, and raw only for Moirai-2. The chosen pools are evaluated on 56 disjoint series per model. Chronos-2 retains a 15.08% MAE reduction versus default, close to the full pool's 15.16%. This model–view profile gives a practical choice based on measured forecasting performance; seasonality alone does not determine the choice, as the M_DENSE controls demonstrate. Sections 3.2 and 5 describe how to select candidates on earlier history, validate their aggregate on later history, and retain a compact pool when its accuracy and cost justify reuse.

The strongest positive evidence is concentrated in Moirai ... The remaining models show considerably smaller or inconsistent improvements.

Response: The expanded evaluation shows gains beyond Moirai-1. We evaluate seven models on all 97 GIFT-Eval tasks, including Chronos-2, Moirai-2, and Toto-2. All seven improve aggregate MAE and MASE over historically selected raw top-3. Removing Moirai-1 also preserves aggregate MAE gains over both raw references:

EvaluationModel–task pairsMAE reduction vs original defaultMAE reduction vs raw-only top-3
All seven models6792.10%4.76%
Excluding Moirai-15820.56%3.40%

Reductions use geometric means of paired task-error ratios. The default reference measures improvement over the designated inference configuration; raw-only top-3 controls for historical context selection and ensembling. Gains vary across models and tasks, and TimesFM and Toto remain slightly worse than default in aggregate MAE. Section 4.1/Tables 1–2 and Appendix E.2 report both references across the full benchmark.

AuthorsOpenReview ↗

Response to Reviewer SHJ7 (3/3)

... Different sampling budgets lack directly measured runtime comparisons, and a substantial portion of the multi-view overhead comes from historical calibration rather than online prediction.

Response: We now measure sampling budgets directly and include pilot, calibration, and online prediction costs when evaluating compact view pools. For Moirai, independent N=20 and N=128 calls use fixed historical patch choices, three seeds, and seven source panels. Increasing the budget reduces median-source MAE by 4.24% using interpolated medians, with online runtime ratios of 0.996–1.027. Patch-discovery cost is reported separately. Sampling can therefore remain useful at low marginal cost in this implementation; the original sub-0.6% finding applies specifically to N=64→128 on the eight original OT series (Appendix E.7).

For multi-view inference, a compact pool can retain the accuracy gain while reducing the measured calibration and prediction time, including its pilot cost. We execute a Solar workload with 56 evaluation series and 19 forecast origins per model:

ModelChosen poolMAE reduction vs default: full → compactTime in seconds: full → pilot + compact
Chronos-2Raw + cycle15.16% → 15.08%86.33 → 67.50
Moirai-2Raw only0.17% → 0.29%79.58 → 58.01

These are time reductions of 21.80% and 27.11% relative to full-pool inference. Online prediction alone also becomes faster, by 19.06% and 14.95%, respectively. Moirai-2 saves work by pruning transformed candidates; Chronos-2 retains the useful cycle view. Comparisons use the same GPU within each model and include transformations and online aggregation, excluding loading, warm-up, data reading, and selector CPU work (Section 4.5/Figure 3; Appendix E.5).

Across the broader five-model, seven-source profiling study, historical pool selection retains 84.6% of the full pool's MAE point-estimate gain with 17.4% fewer prediction branches, including pilot and calibration work. These branch counts are distinct from the measured runtimes above. Fixed raw+cycle achieves similar aggregate accuracy with fewer branches, and the source-bootstrap intervals for both full and historically chosen pools versus default include zero (Table 3; Appendix E.5).

The smaller Solar workload, with eight series and two origins, incurs higher total time because of pilot overhead. The two workloads show why the intended amount of reuse matters: a pool should be validated for accuracy and evaluated on the combined pilot, calibration, and forecasting cost of its intended workload.

Updated paper. We have updated the paper, with key additions highlighted in yellow. The full benchmark data and results are available in the linked anonymous repository.


We hope these clarifications and revisions address your concerns, and we would be happy to discuss any remaining questions.

Reviewer_rHesOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies test-time scaling for frozen time series foundation models. It separates three sources of gain: repeated sampling at a fixed input, multi-view inference (raw, cycle-residual, trend-residual, subseries), and history-based selection/aggregation. It evaluates Chronos, Sundial, Moirai and TimesFM on 8 datasets, with MAE as primary metric and raw top-k / Cycle+AR / simple pools as controls. The main message is cautionary: sampling saturates quickly (64->128 <0.6%), multi-view top-3 can beat raw top-k (e.g. Moirai -10% on 16 Electricity) but the number collapses once you change the raw reference (e.g. -22.48% vs all-raw but -0.28% vs raw top-k on 4 Traffic), history winners often reverse at test time, and larger pools / calibration cost can wipe out the gain.

Critical strengths

The scope of this work is generally novel, and the three-way split, sampling vs views vs aggregation, sounds reasonable. The paper compares against sufficiently sampled, historically selected raw baselines instead of a weak single-context baseline. The component controls (Cycle+AR) and the cost breakdown including calibration/probes are also welcome, while most prior work only reports the candidate potential.

Critical weaknesses

My main concern is that as an empirical paper without a method, the takeaways feel fragmented and not yet actionable.

First, we learn a lot about what does not work, but little about what to do next. After RQ1-7 I still wonder giving a new model/series, where should I start? When is cycle likely to help vs hurt? The paper shows Moirai cycle is -4.05% on 8 OT but +0.13% on 48 additional, and TimesFM cycle+TSFM beats AR by -28% yet loses to raw by +17% on Traffic. That's interesting, but it stops at "it depends on model/task/selector" without distilling any predictive pattern.

Second, the evaluation is still narrow for a "systematic" claim, and leakage is unaudited. ETT datasets are exactly the series most likely to be in pretraining, and the authors acknowledge overlap was not checked. With mainly H=96 univariate point forecasts and 4 checkpoints, it's hard to tell whether the reported gains/losses are about test-time scaling in general or about these specific old benchmarks.

Third, the history-transfer problem is diagnosed but not addressed. The oracle vs history gap is 20-30% vs ~5%, 13/21 TimesFM wins reverse, expanding Moirai pool 20->28 makes Traffic +5.28% worse. I agree this is an important negative result, but the paper leaves it as "reliable selection remains future work" without even a preliminary attempt at when transfer fails.

Overall this is a solid reality check, but in its current form it reads more like a careful technical report than an ICLR paper: strong on controls, thin on generalizable insight or guidance for practitioners.

Questions for authors

Please refer to the weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer rHes (1/3)

Thank you for recognizing the value of the controlled comparisons. Our work introduces a structured multi-view framework for test-time scaling with frozen TSFMs, combining alternative temporal representations with historical selection and compact aggregation for both sampling-based and direct-quantile models. Experiments across seven TSFMs and all 97 GIFT-Eval tasks demonstrate aggregate gains over default inference and historically selected raw ensembles. Controlled ablations distinguish the contributions of view generation, selection, and aggregation. We further develop and evaluate a historical profiling procedure that selects compact view pools, retaining most of the full pool's gain with fewer prediction branches while accounting for profiling costs.

... the takeaways feel fragmented and not yet actionable.

Response: The revised paper connects the findings through the process of improving a frozen TSFM with structured input views. We first establish gains beyond raw-input ensembling and identify which representations contribute them (Sections 4.1–4.2). We then show how historical selection and compact aggregation turn these candidates into more accurate forecasts (Section 4.3). Finally, historical profiling examines how much of the gain can be retained with fewer prediction branches, supported by measured runtime comparisons (Sections 4.4–4.5).

... giving a new model/series, where should I start?

Response: For a new model and series, Section 5 ("Using a new model and dataset") gives the following guidance:

  1. Fix the forecast horizon and target metric, retaining default raw and raw-only top-3 as references.
  2. Compare raw, raw+cycle, raw+trend, and raw+subseries on completed historical windows. Select candidates on earlier windows and evaluate their top-3 aggregate on later windows.
  3. Reuse a compact pool when its validation accuracy and computational cost justify it; retain raw when the alternatives do not improve the target metric. Count pilot, calibration, and prediction costs over the intended workload.

When several related series are available, pilot histories can guide the pool choice; each evaluation series still ranks candidates using its own history. Section 3.2 specifies this historical evaluation procedure.

The main paper illustrates pool selection on Solar (Section 4.4, Table 4, p. 9). Eight pilot series choose a pool, which is then evaluated on 56 disjoint series per model. Chronos-2 selects raw+cycle and retains a 15.08% MAE reduction versus default, close to the full pool's 15.16%. On the same data, TimesFM selects raw+trend, while Moirai-2 selects raw only. Historical evaluation thus produces different view choices for different models on the same source.

Across the broader profiling study, we evaluate five models on seven source panels: 35 model–source pairs and 1,745 model–evaluation-series pairs, using eight pilot series per source. The main paper reports this comparison in Section 4.4/Table 3. All strategies below use historical top-3; positive reductions indicate improvement.

Pool strategyMAE reduction vs defaultPrediction branches saved vs full pool
Full multi-view pool2.23%—
Pool chosen from pilot histories1.88%17.4%
Fixed raw+cycle1.85%32.5%

The chosen pools retain 84.6% of the full pool's aggregate MAE point-estimate gain, including pilot and calibration work in the branch accounting. Fixed raw+cycle is a strong, cheaper reference with similar aggregate accuracy. Both full and historically chosen pools have source-bootstrap intervals versus default that include zero, so the retention ratio describes this study's point estimates.

AuthorsOpenReview ↗

Response to Reviewer rHes (2/3)

When is cycle likely to help vs hurt? ... without distilling any predictive pattern.

Response: The new Moirai ablation identifies a specific source of improvement. On the original 48 additional series, the full pool reduces MAE by 7.42% over raw top-k Median. Raw+cycle retains a 7.67% reduction, whereas removing cycle leaves only 0.42%. These comparisons match historical scoring, aggregation, and available ensemble size. Thus, cycle candidates account for most of this gain.

A compact raw+cycle pool improves Moirai beyond raw-input ensembling. On the 48 additional series, selecting a single candidate from this pool reduces MAE by 4.52% relative to the historically selected raw top-k ensemble. Aggregating the top three candidates increases this reduction to 7.67%. Historical selection identifies useful forecasts from the expanded pool, and compact aggregation further improves their accuracy.

Cycle forecasts are selected for 46 of the 48 series in the full multi-view pool; for 24 series, all selected forecasts come from cycle views. Keeping raw candidates available also matters: selecting a single forecast from cycle candidates alone increases MAE by 0.88% against the same raw reference.

A phase control gives a more specific explanation in another model: on a fixed Chronos-2 KDD panel, aligned cycle reduces MAE by 9.50% versus raw and 5.26% versus a half-cycle shift, while centering and scale matching have approximately zero effect. This supports the value of temporal alignment in that panel. Strongly periodic M_DENSE series nevertheless deteriorate under cycle in all three tested models. Together, these results motivate testing the model's response to a representation, rather than choosing a transformation from seasonality alone.

... the history-transfer problem is diagnosed but not addressed. ... without even a preliminary attempt at when transfer fails.

Response: We evaluate two practical ways to reduce dependence on a single historical winner. Across all 679 GIFT-Eval model–task pairs, multi-view top-3 reduces aggregate MAE by 1.90% versus historical single selection and 3.22% versus averaging the entire pool. Both aggregate comparisons favor top-3 in every model.

A separate control mixes default raw and multi-view top-3 with a fixed 1:1 weight. It retains 94.1% of the aggregate MAE gain while reducing the number of pairs with more than 5% MAE degradation from 59 to 19. These profiling and mixture analyses are retrospective; the disjoint-series evaluation tests transfer to other series within the studied sources. They provide concrete alternatives to relying on one historical winner, while future ranking changes remain possible.

... the evaluation is still narrow ... and leakage is unaudited. ETT datasets ... most likely to be in pretraining ... mainly H=96 univariate point forecasts and 4 checkpoints ...

Response: We have expanded the evaluation from four models on eight datasets to seven models on all 97 GIFT-Eval tasks (679 model–task pairs), covering seven domains and forecast horizons from 6 to 900. Multi-view top-3 improves aggregate MAE over historically selected raw ensembles in all seven models, including Chronos-2, Moirai-2, and Toto-2.

We also compare short, medium, and long forecasts on the same 21 dataset–frequency configurations and seven models, giving 147 model–task pairs at each horizon (Appendix E.2). Multi-view top-3 improves MAE over both raw references at all three horizons, with larger aggregate gains at medium and long horizons in this matched comparison. Reductions below use geometric means of paired task-error ratios.

Forecast horizonMAE reduction vs original defaultMAE reduction vs raw-only top-3
Short1.73%4.60%
Medium2.25%8.41%
Long2.71%7.83%

Removing all ETT tasks preserves the gains. The remaining 81 tasks give 567 model–task pairs, with MAE reductions of 5.07% over raw-only top-3 and 2.01% over the original default. Aggregate MAE gains over both references also remain positive when each of the 19 source families is excluded in turn (Appendix E.2). The observed improvement therefore persists beyond ETT and does not depend on retaining any single source family.

Regarding pretraining overlap, Appendix E.8 documents the available public training-data disclosures; an independent sample-level overlap audit remains incomplete. The exclusion analyses assess whether gains depend on particular sources, not whether all evaluation sources were unseen during pretraining. Separately, all GIFT-Eval calibration targets are observed before testing, and historical choices remain fixed over each test block. These chronology controls address future-label use during selection. Our evaluation concerns independent univariate point forecasts from frozen models.

AuthorsOpenReview ↗

Response to Reviewer rHes (3/3)

Updated paper. Sections 3.1–3.2 specify the framework and historical selection procedure. Sections 4.1–4.5 now connect cross-model gains (Tables 1–2), attribution (Figure 2), compact aggregation, pool selection (Tables 3–4), and actual prediction cost (Figure 3). Section 5 gives the practitioner procedure; Appendix E.5 reports all 35 profile outcomes, and E.6 reports the fixed-mixture control. Major additions are highlighted in yellow.


We hope these clarifications and revisions address your concerns, and we would be happy to discuss any remaining questions.

AuthorsOpenReview ↗

Follow-up Comment to Reviewer rHes

Thank you again for your thoughtful feedback. As the discussion deadline approaches, we would appreciate it if you could take a look at our rebuttal and revised manuscript. Please let us know if you have any remaining concerns or questions. We would be happy to clarify any points or discuss them further.

Reviewer_EofEOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the test time scaling laws in time series foundational models (similar to LLMs). They study two approaches: i) single-view sampling, ii) multi-view sampling. They show that repeated sampling improves point forecasts with saturations reaching at 64 to 128 samples. Similarly multi-view aggregation achieves an MAE reduction with gains dependent on the model and task.

Critical strengths

The paper proposes a very interesting problem of scaling in TSFMs and uses a sound experimental setup to come up with very useful findings.

Critical weaknesses

The study only covers 4 TSFMs although there are several strong TSFMS - Toto-2, Timer-S1, PatchTST, newer Moirai models. Also, not sure why they used older versions of Moirai and Chronos. Given that the paper is about studying test-time scaling, expanding the list of models is critical to show generalizability of findings.

Further, the benchmarks should be more comprehensive given the nature of the paper (why not entire GIFT-Eval). Also, does the improvement happen on metrics such as MASE and MAPE as well? It’s not clear in the paper.

Questions for authors

Can you evaluate more recent TSFMs?

4 comments
AuthorsOpenReview ↗

Response to Reviewer EofE (1/2)

Response to Reviewer EofE

Thank you for the specific requests on newer models, complete benchmark coverage, and additional metrics. We have completed an evaluation of seven TSFMs on all 97 GIFT-Eval tasks, reporting MAE, MASE, MAPE, and sMAPE. The results support multi-view inference across both stochastic and direct-quantile models.

The study only covers 4 TSFMs ... not sure why they used older versions of Moirai and Chronos.

Response: Thank you for raising this point. Our original model selection aimed to cover complementary pretraining formulations, training objectives, and forecast-generation mechanisms relevant to test-time scaling. The original evaluation already included TimesFM-3, a recent model reporting state-of-the-art results on GIFT-Eval and FEV-Bench.

Chronos-T5 uses autoregressive next-token prediction over discretized values with a cross-entropy objective; Sundial learns continuous next-patch distributions through its flow-matching TimeFlow objective; Moirai-1 uses masked forecasting with a mixture-distribution negative log-likelihood; and TimesFM-3 uses contiguous patch masking (CPM) and provides deterministic quantile forecasts in our evaluation.

These differences matter for how additional inference produces candidate forecasts. Chronos-T5, Sundial, and Moirai-1 support sampling through different mechanisms, allowing us to study repeated sampling at a fixed input. TimesFM-3 provides a complementary setting in which repeated calls on an unchanged input do not produce new stochastic candidates. Together, these models allow us to investigate both within-input sampling and structured multi-view inference, which generates alternatives by changing the input representation. Retaining the original Chronos and Moirai models preserves these distinct sampling mechanisms alongside their newer generations.

We agree that evaluating newer models is important for assessing the breadth of the findings. We have therefore added Chronos-2, Moirai-2.0-R-Small, and Toto-2.0-313M, and evaluated all seven models on the complete 97-task GIFT-Eval benchmark. This extends the study across model generations while retaining the original comparisons of sampling and multi-view inference. Exact model revisions and inference settings are specified in Appendix E.1.

The table evaluates both the gain over the original model's default inference and the additional value of structured views under the same historical top-3 selection and aggregation rule. It reports multi-view top-3 Mean on all 97 tasks per model; positive MAE reductions indicate improvement, and negative values indicate deterioration.

ModelMAE reduction vs original defaultMAE reduction vs raw-only top-3Tasks improved
Chronos-T52.56%4.53%68/97
Sundial1.38%0.27%46/97
Moirai-110.86%12.53%73/97
TimesFM 3.0−1.36%9.53%66/97
Chronos-20.04%2.08%63/97
Moirai-21.28%1.22%57/97
Toto-2−0.63%2.47%63/97
All 679 model–task pairs2.10%4.76%436/679

Improvement counts are relative to raw-only top-3. Original default uses one raw-input configuration, with 20-sample P50 for stochastic models or native P50 for direct-quantile models. MAE reductions are computed from geometric means of paired task-error ratios.

All seven models improve over historically selected raw ensembles, including the three newly added models. Each added model also improves MAE on a majority of the 97 tasks against this reference. Five models improve over original default inference; the TimesFM and Toto exceptions are visible in the table. Timer-S1 and PatchTST remain outside the evaluated model set.

AuthorsOpenReview ↗

Response to Reviewer EofE (2/2)

... the benchmarks should be more comprehensive ... (why not entire GIFT-Eval).

Response: We have completed a full GIFT-Eval extension during revision, evaluating all seven models on all 97 tasks: 679 model–task evaluations, comprising 388 for the original four models and 291 for the three newly added models. Across all official test series and forecast origins, this covers 2,599,310 test forecasting instances. Each instance is a model–task–series–forecast-origin combination, counted once across the compared methods and metrics.

The extension includes 55 short-, 21 medium-, and 21 long-horizon tasks across 55 dataset–frequency configurations and seven domains, with horizons from 6 to 900. This expands the original eight-dataset, predominantly H=96 study to the complete benchmark. All models use the official test series and windows. Appendix E.9 provides the complete task-level results against both raw references.

For each series, up to four completed pre-test windows rank candidate forecasts; the ranking is fixed across its test block. All models use independent univariate inference. Stochastic raw and transformed branches each use 20 samples summarized by P50; direct-quantile models use native P50. Thus, the raw reference already benefits from within-input sampling where available. Multi-view changes the representations supplied to the model, providing additional candidate forecasts for either output interface.

The original controlled experiments and the full benchmark retain their respective protocols and aggregation statistics. On the same 21 dataset–frequency configurations available at all three horizon lengths, improvements persist at short, medium, and long horizons (Appendix E.2).

... does the improvement happen on metrics such as MASE and MAPE as well?

Response: The gains extend beyond MAE. MASE improves against both raw references; relative to raw-only top-3, improvements hold for all seven model aggregates and 500/679 tasks (73.6%). The table summarizes multi-view top-3 on the complete benchmark.

MetricReduction vs original defaultReduction vs raw-only top-3Models improvedTasks improved
MAE2.10%4.76%7/7436/679
MASE2.06%6.12%7/7500/679
MAPE−2.88%2.76%5/7394/679

Improvement counts are relative to raw-only top-3. Positive reductions indicate improvement; reductions use geometric means of paired task-error ratios over all 679 model–task pairs.

MAPE improves over raw-only top-3 but worsens relative to the original default. Our configurations are selected by MAE on pre-test historical windows, and these selections are held fixed when evaluating all metrics. The selector therefore does not optimize historical MAPE, and its choices need not improve test MAPE over the default.

To examine the effect of the selection metric, we reranked the cached candidates by historical MAPE on our previously fixed panel of five models and seven tasks (35 model–task groups; 1,745 model–series combinations). Both raw-only and multi-view top-3 used the new criterion, with the same candidate predictions, calibration and test windows, and averaging rule. Relative to MAE-based selection, multi-view test MAPE decreased by 1.07%, with improvements in 20/35 groups, while MAE and MASE increased by 0.29% and 0.64%. The largest model-level MAPE reduction was 12.11% for Moirai-1; two of the five model aggregates improved.

On this supplementary panel, the MAPE increase over the original default narrowed from 10.82% to 9.63%. Multi-view MAPE remained 2.90% higher than raw-only top-3 when both used MAPE-based selection. Changing the selection objective helped in some settings, with a small cost in MAE and MASE, but did not close the MAPE gap to the default. These panel results are separate from the 679-pair benchmark above and are now reported in Appendix E.2 (Table 25). Appendix E.1 documents zero-denominator handling, and Appendix E.2 also reports the complete benchmark's model-level metrics, including sMAPE and the near-zero-target MAPE diagnostic.

Updated paper. We have updated the paper. The full benchmark data and results are available in the linked anonymous repository.


We hope these clarifications and revisions address your concerns, and we would be happy to discuss any remaining questions.

I thank the authors for the additional experiments. It addresses my concerns. Overall, a very interesting read and well written paper. Increased my score to 3.

AuthorsOpenReview ↗

Response to Reviewer EofE

Thank you for your positive feedback and for increasing your score. We sincerely appreciate your constructive suggestions, which have helped us substantially improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.