Structural Modes of Context-to-Context Attention in Chronos: A Mechanistic Autopsy of a Time Series Foundation Model

Foundation models for time series forecasting are trained on vast, high-dimensional real-world data and synthetic corpora, yet they can fail on zero-shot forecasting of sequences generated by simple, deterministic functions. In this work, we investigate the internal mechanism behind this shortcoming of the original Chronos, a time series foundation model whose pointwise tokenization makes its attention directly interpretable. Following techniques in explainable AI, we probe Chronos with the controlled stimuli of Jacobi elliptic functions, auto-regressive iterative series, and the logistic map as inputs, where we vary the parameters that modulate periodicity, locality, and chaoticity. We extract the average attention matrices of each encoder block. These matrices resolve into a small set of canonical structural modes. Early blocks are dominated by value-based modes, in which tokens with equal values attend to each other and all tokens attend to extrema, while later blocks concentrate attention on local neighborhoods and off-diagonal bands. We quantify these modes with locality and periodicity metrics defined on the attention matrices, and we relate the behavior of the encoder to forecast accuracy through rank correlations and layer pruning. For several stimulus families, the forecast accuracy improves when the encoder resolves the input into at least one interpretable structural mode of strong locality or clear periodicity, and degrades when it resolves neither, as for near-aperiodic or chaotic signals. We also address how our analysis can be extended to other time series foundation models to diagnose how models encode using other architectural strategies such as patch embedding and causal masking. This suggests a promising direction for interpreting how deep learning architectures structure data processing.