Does More History Help? Understanding Context Length in Time-Series Foundation Models
Abstract
*Time series foundation models* (TSFMs) for time series forecasting are designed to operate on variable input context lengths. This leads to the question: *how much context is worth using for a given task?* While the current trend favors longer contexts to enable more flexible predictors, their beneficial effect on inference-time accuracy is not guaranteed. In this work, we systematically investigate the influence of context length on forecasting performance across state-of-the-art TSFMs, and show consistent diminishing returns as the context length grows: an effect called *context saturation*. Through experiments in controlled environments, we provide insight into the factors driving context saturation, and show signs of recency bias in TSFMs, where their ability to leverage long-range dependencies varies substantially by model. Furthermore, through oracle context-length selection, we demonstrate the existence of operating points with both lower aggregate forecasting error and lower computational cost than those resulting from maximum context length. Finally, we propose a methodology for *zero-shot* inference-time context length selection, leveraging only synthetic data. Evaluated across all TSFMs on the GiftEval benchmark, our approach achieves substantial efficiency gains, validating the practical impact of our findings.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.