Finding the Zeitgeist in Time Series Foundation Models

Time series foundation models (TSFMs) achieve strong zero-shot and transfer performance across diverse tasks, yet their internal representations remain poorly understood. Although sparse autoencoders (SAEs) have emerged as a powerful tool for mechanistic interpretability in language and vision models, this has not yet been established with TSFMs. We conduct the first large-scale exploration of their internal representations using SAEs, uncovering sparse features that align with interpretable temporal patterns. Moving beyond only probing for hypothesized concepts, SAEs cast opaque model internals into disentangled, often monosemantic features. To structure their large latent space, we establish *Zeitgeist*, an unsupervised method for fully automated characterization of individual latent features. Zeitgeist enables creating a characterizing atlas of activating geometries, captions, frequency, locality, and positional affinity, describing the inner workings of TSFMs. We demonstrate Zeigeist's generality across 9 TSFMs, spanning 3 architectural families, and provide new insights into their systematic differences in reasoning. Moreover, we show that Zeitgeist's internal representations are not only a tool for interpretability but can also be leveraged for targeted interventions on a TSFM's activations, providing a strong foundation for steering downstream applications.