acceptodds
Under review as a conference paper at ICLR 2027

What Do Time-Series Foundation Models Share? An Input-Kernel Accounting of Representational Agreement

Abstract

Do time-series foundation models organize input histories in a shared way? We compare the matrices of pairwise history similarities produced by ten variants from seven families on 17 datasets. Released pretrained weights (PT) agree more than independent architecture-matched initializations (INIT). On the full inventory, more of the additional agreement lies in the input kernels for mean pooling than for last-position pooling. The last-position residual stays positive across depth, reference expansion and family deletion. The mean-linear endpoint interval crosses zero; mean-RBF and intermediate mean-linear depths stay positive. On later Wikipedia records the mean-linear residual is larger than the last-linear residual. The last-linear remainder is a difference in residual direction: PT and INIT have similar residual norms, and permuting each released parameter tensor removes the additional agreement. Small MLP and CNN predictors that improve within-dataset forecasting also develop additional last-position agreement outside the same input span. The balance between the input span and the residual depends on the readout and the population.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.