- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This study shows how time-series foundation models change when given longer input histories. The authors test six pretrained models on controlled Gaussian processes where the true one-step predictive distribution is known. Their main approach uses conditional resampling to keep the predictive mean fixed, and sometimes the last observation as well, so that forecast error can be separated into different components. The experiments include two-lag and three-lag settings together with autoregressive baselines. The main result is that longer histories can make forecasts more stable without necessarily improving how the models respond to the true predictive signal. The three-lag experiment also shows different response changes for Chronos-Bolt and Moirai. The study is careful, but most of the evidence comes from synthetic linear-Gaussian processes and one-step forecasting. Real-world benchmark validation is limited, and the related work is relatively light.
Critical strengths
- By resampling histories while fixing the predictive mean, and then additionally the last observation, the paper separates forecast variation from errors in the model's average response. The underlying tools are established, but this use of them provides a more informative view of TSFM behavior than forecast accuracy alone.
- The paper is also careful in how it handles the statistical analysis. It distinguishes exploratory and post hoc findings from the later prospective three-lag test, and checks the finite-sample behavior of several estimators through simulation. This makes the reported effects more credible, although the experiments themselves still cover a fairly narrow setting.
Critical weaknesses
- While the conditional-resampling framework is useful, much of its mathematical foundation comes from established statistical methods. The paper shows how these tools can be applied to TSFMs, but I am not fully convinced that the analysis leads to sufficiently new insights beyond what existing approaches can reveal. In particular, the main response bias can already be identified through standard regression. The authors need to make a stronger case for what the proposed decomposition adds to our understanding of model behavior. This is important when assessing the paper's methodological novelty and overall significance.
- The findings are still limited to a relatively small set of linear-Gaussian processes. I understand why controlled synthetic data is necessary for the proposed analysis, but I do not know why there is a lack of real-world benchmarks. The results vary noticeably across process conditions, and the prospective three-lag experiment remains within a closely related setting. It is therefore difficult to tell how general the observed response patterns are, particularly whether they reflect broader properties of the tested checkpoints or depend mainly on the chosen processes. This limits the significance of the empirical findings.
Questions for authors
- The paper shows that conditional resampling can separate history variation from nonlinear conditional-mean errors, which standard regression cannot distinguish. However, several main response findings can already be obtained through regression. Which empirical conclusions specifically depend on the additional decomposition, and how do they change our understanding of TSFM behavior?
- The main experiments focus on two-lag and three-lag linear-Gaussian processes. While this controlled setting is reasonable for the proposed analysis, it remains unclear how much the findings depend on these assumptions. Is there any evidence or theoretical argument suggesting that the response patterns would also hold for nonlinear or non-Gaussian processes?
- The results show noticeable variation across process conditions. For example, Appendix C.1.1 reports substantially different response changes for Bolt depending on the local coefficient share . Given that the bootstrap keeps the conditions fixed, how robust are the overall Bolt–Moirai contrasts to different parameter ranges or condition weightings?
- In Appendix D.1, Table 11 suggests that most of the remaining regression residual comes from history variation. However, the residual-share bounds are exploratory, and their coverage has not been separately calibrated. How much confidence should be placed in this conclusion, particularly for Bolt and Moirai, where the upper bounds are relatively large?
- The two-lag confirmation experiment supports only three of its six prespecified endpoints (Table 4), while the later three-lag test supports all three selected response hypotheses (Table 3). Since these experiments test different hypotheses, could the authors clarify which conclusions are supported by each and how the earlier incomplete confirmation affects the overall interpretation?
- The related-work discussion is relatively limited given the paper's focus on TSFM behavior and context-length effects. Could the authors better explain how their work differs from existing studies on model response, inductive biases, and statistical diagnostics? In particular, what constitutes the main methodological or empirical novelty?
I initially thought this was mainly a theoretical paper, and I appreciated the careful statistical analysis and experimental design. However, after reading it more closely, I found that much of the framework relies on established statistical methods. The idea of applying these methods to TSFMs is interesting, but I am not convinced that it provides enough new insight. If the contribution is mainly empirical, the experiments are still largely limited to linear-Gaussian processes, with little evidence that the main findings extend beyond these settings. I understand why synthetic data is used here, but this leaves me unsure of the broader significance of the results. The limited related-work discussion also makes it harder to judge what is genuinely new.