Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Models
Abstract
The recent surge of interest in Time Series Foundation Models has rapidly advanced the field. Yet, heterogeneous training setups across studies make it difficult to determine whether performance improvements came from architectural innovations or effective data engineering. In this work, we investigate whether a standard patch Transformer architecture can achieve state-of-the-art zero-shot forecasting performance when paired with a straightforward training protocol. We conduct a comprehensive ablation study covering model scaling, data composition, and training techniques to isolate the essential ingredients for high performance. Our findings identify the key drivers of forecasting accuracy, confirming that the generic architecture itself can deliver high performance with good scalability. By strictly controlling these experimental variables, we provide comprehensive empirical results on model scaling across multiple dimensions. Furthermore, we establish that the effectiveness of this training protocol generalizes robustly across benchmarks and alternative architectures. We release our open-source model and detailed findings to establish a transparent, reproducible baseline for future research.
Related papers
See more related submissions when the market opens.