Test-time State-augmented Adaptive Forecasting for Time Series Foundation Models

Time series foundation models (TSFMs) have demonstrated zero-shot forecasting capability on unseen time series data from various domains. When forecasting with a TSFM, a fixed-length input window is fed into the TSFM. Since typical TSFMs are transformer-based, they incur high computational costs when the input window length is large, due to the transformers' quadratic computational complexity with respect to the input length. Although shortening the input window length can save computational costs, there is a trade-off between the computational costs and forecast performance because transformers are stateless and cannot capture temporal dependencies beyond an input window, and short input windows make a distributional gap relative to the TSFM's pre-training. In this paper, we address the trade-off between the computational cost and performance when forecasting with a TSFM. To mitigate accuracy drops with short input windows, we propose State-augmented Adaptive Forecasting (SAAF), which enables off-the-shelf pre-trained TSFMs to capture temporal dependencies beyond the input window under restricted window lengths. During inference, SAAF incorporates a stateful module that captures temporal dependencies with short input windows. In addition, SAAF transforms a TSFM's outputs to mitigate the TSFM's performance degradation caused by short input windows. Experimental results show that SAAF had lower forecast errors than baselines, especially with short input windows, across various forecasting benchmarks, at lower computational costs.