acceptodds
Under review as a conference paper at ICLR 2027

Beyond Forecast Accuracy: How Time-Series Foundation Models Respond to Longer Histories

Abstract

Longer histories can stabilize forecasts without correcting their average response to predictive information. We study this distinction in six frozen time-series foundation model checkpoints on Gaussian processes with known next-step distributions. At 2048 observations, the five deterministic checkpoints respond too weakly to the true predictive mean on average. Their excess mean squared errors are 24–45% of predictable variance, compared with 6.5% for a fitted AR(64) baseline. Yet Chronos-Bolt and Moirai vary less across histories with the same predictive mean and last observation than that baseline. Linear regression exposes the response bias. Conditional resampling then separates two sources of the regression residual: a curved average response and variation across matched histories. Exploratory bounds suggest that most residual energy comes from the latter. History length also changes the association with the last observation at a fixed predictive mean. Following exploration, a prospective three-lag test supports opposite Bolt–Moirai changes from 512 to 2048, averaged over 16 conditions; Moirai's last-observation response varies nonmonotonically with history length. These comparisons show why greater forecast stability need not repair the main sources of error.

Primary area:
learning on time series and dynamical systems
Keywords:
time series foundation models, context length, conditional resampling, forecast stability, predictive response

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_TxGTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This study shows how time-series foundation models change when given longer input histories. The authors test six pretrained models on controlled Gaussian processes where the true one-step predictive distribution is known. Their main approach uses conditional resampling to keep the predictive mean fixed, and sometimes the last observation as well, so that forecast error can be separated into different components. The experiments include two-lag and three-lag settings together with autoregressive baselines. The main result is that longer histories can make forecasts more stable without necessarily improving how the models respond to the true predictive signal. The three-lag experiment also shows different response changes for Chronos-Bolt and Moirai. The study is careful, but most of the evidence comes from synthetic linear-Gaussian processes and one-step forecasting. Real-world benchmark validation is limited, and the related work is relatively light.

Critical strengths
  1. By resampling histories while fixing the predictive mean, and then additionally the last observation, the paper separates forecast variation from errors in the model's average response. The underlying tools are established, but this use of them provides a more informative view of TSFM behavior than forecast accuracy alone.
  2. The paper is also careful in how it handles the statistical analysis. It distinguishes exploratory and post hoc findings from the later prospective three-lag test, and checks the finite-sample behavior of several estimators through simulation. This makes the reported effects more credible, although the experiments themselves still cover a fairly narrow setting.
Critical weaknesses
  1. While the conditional-resampling framework is useful, much of its mathematical foundation comes from established statistical methods. The paper shows how these tools can be applied to TSFMs, but I am not fully convinced that the analysis leads to sufficiently new insights beyond what existing approaches can reveal. In particular, the main response bias can already be identified through standard regression. The authors need to make a stronger case for what the proposed decomposition adds to our understanding of model behavior. This is important when assessing the paper's methodological novelty and overall significance.
  2. The findings are still limited to a relatively small set of linear-Gaussian processes. I understand why controlled synthetic data is necessary for the proposed analysis, but I do not know why there is a lack of real-world benchmarks. The results vary noticeably across process conditions, and the prospective three-lag experiment remains within a closely related setting. It is therefore difficult to tell how general the observed response patterns are, particularly whether they reflect broader properties of the tested checkpoints or depend mainly on the chosen processes. This limits the significance of the empirical findings.
Questions for authors
  1. The paper shows that conditional resampling can separate history variation from nonlinear conditional-mean errors, which standard regression cannot distinguish. However, several main response findings can already be obtained through regression. Which empirical conclusions specifically depend on the additional decomposition, and how do they change our understanding of TSFM behavior?
  2. The main experiments focus on two-lag and three-lag linear-Gaussian processes. While this controlled setting is reasonable for the proposed analysis, it remains unclear how much the findings depend on these assumptions. Is there any evidence or theoretical argument suggesting that the response patterns would also hold for nonlinear or non-Gaussian processes?
  3. The results show noticeable variation across process conditions. For example, Appendix C.1.1 reports substantially different response changes for Bolt depending on the local coefficient share rr. Given that the bootstrap keeps the conditions fixed, how robust are the overall Bolt–Moirai contrasts to different parameter ranges or condition weightings?
  4. In Appendix D.1, Table 11 suggests that most of the remaining regression residual comes from history variation. However, the residual-share bounds are exploratory, and their coverage has not been separately calibrated. How much confidence should be placed in this conclusion, particularly for Bolt and Moirai, where the upper bounds are relatively large?
  5. The two-lag confirmation experiment supports only three of its six prespecified endpoints (Table 4), while the later three-lag test supports all three selected response hypotheses (Table 3). Since these experiments test different hypotheses, could the authors clarify which conclusions are supported by each and how the earlier incomplete confirmation affects the overall interpretation?
  6. The related-work discussion is relatively limited given the paper's focus on TSFM behavior and context-length effects. Could the authors better explain how their work differs from existing studies on model response, inductive biases, and statistical diagnostics? In particular, what constitutes the main methodological or empirical novelty?


I initially thought this was mainly a theoretical paper, and I appreciated the careful statistical analysis and experimental design. However, after reading it more closely, I found that much of the framework relies on established statistical methods. The idea of applying these methods to TSFMs is interesting, but I am not convinced that it provides enough new insight. If the contribution is mainly empirical, the experiments are still largely limited to linear-Gaussian processes, with little evidence that the main findings extend beyond these settings. I understand why synthetic data is used here, but this leaves me unsure of the broader significance of the results. The limited related-work discussion also makes it harder to judge what is genuinely new.

Reviewer_U6baOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work aims to investigate the relationship between forecast accuracy and forecast stability in time series foundation models as we change the amount of context provided to the model, arguing that stable forecasts do not necessarily mean more accurate forecasts. With that aim, it uses several time series foundation models evaluated on autoregressive processes with a memory of 64.

Critical strengths

None.

Critical weaknesses

The work mentions that is evaluated on six TSFMs but then the evaluations are rarely perform on all of them, and instead just focus on 2 (outdated) models.

The main contribution of the paper is heavily unclear, and following what can be understood, it seems the claims can be summarized by just Table 2 of results.

Authors make claims based on the forecast of only one different type on input. This is not enough evidence to claim anything about the behaviour or mechanisms of this or any type of model.

In general, the paper is really badly written and unclear. The main aims, research questions or contributions are not undestandable, and even if it had interesting or relevant results, its current presentation puts it way below any minimum standart of quality.

Questions for authors

What is the reason to test your claim and your findings in such a limited family of inputs? Specially considering TSFM are supposed to be zero shot forecaster of any time series.

Which reasoning did you use to chose the TSFMs? Why is a set chosen at the beggining of the work, comprising 6 models but then only 2 of them are thoughfully analyzed? In addition, these 2 are the oldest and less performant in benchmarks from such a set.

The work it is mentioned an anonymous package but no link can be found, are those the codes for this work? Are you releasing them?

Reviewer_rphCOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This is an emergency review.

The work investigates how the prediction error of time-series foundation models can be decomposed into different aspects: variance across context values, dependence on the last time step, and the error given the correct mean. It finds that these models are too insensitive to the true predictive mean. While Bolt and Moirai are more stable than an autoregressive baseline, they are less accurate overall. Chronos-2 and TimesFM are both less stable and less accurate. Additionally, the authors observe that Chronos Bolt and Moirai behave differently as context length grows: Chronos Bolt depends less on the last time step, while Moirai depends more.

Critical strengths
  1. Understanding when and why time series foundation models are effective remains an open and important area of research.
Critical weaknesses
  1. The overall clarity of the work could be strongly improved: While the math is largely sound (though the assumptions could be made more explicit), the surrounding prose often does not help with understanding it. The different sections are somewhat unconnected. For instance, among others, Sec. 3 abruptly dives into definitions without any prior motivation. The overall flow makes this work unnecessarily hard to read (for me).
  2. There is almost no connection to the existing literature: There is no dedicated related work section, and Sec. 1 is not an adequate replacement. The connections made in the final discussion in Sec. 7 are too late and superficial.
Questions for authors
  1. Why is Sundial excluded from some analysis (e.g., Fig. 4, right-hand side)?
  2. How does this connect to the many existing findings digging into time series foundation model performance and mechanisms? Does this work confirm, contradict, or complement them?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.