SpecSpin: Spectral Truncation and Rotation for 4-bit Zero-Shot Time-Series Foundation Models

Time-series foundation models (TSFMs) forecast zero-shot across domains, but they are expensive to serve, and 4-bit weight quantization, the standard remedy borrowed from large language models, is not free for them: the autoregressive loop compounds rounding error over long horizons. We show that TSFM attention has spectral structure that a quantizer can exploit. Along the key/time axis, pre-softmax score rows are smooth and recency-biased, so most of their energy sits in the lowest DCT frequencies; along the input-feature axis, the q/k/v projections concentrate their energy in a small set of DCT directions. We introduce (Spectral Spin quantization), a training-free, calibration-free 4-bit recipe built on both observations. truncates each q/k/v projection to its highest-energy DCT directions, applies a Haar rotation inside the kept subspace, quantizes all linear layers to 4-bit NormalFloat (NF4), and low-pass filters every attention score row before the softmax. On three architecturally distinct TSFMs (Chronos, TimeMoE, Moirai), six benchmarks and four horizons, is the best 4-bit method on overall average for all three backbones and lowers average MSE below the BF16 model on each ($-0.9\%$, $-5.5\%$, $-5.2\%$, versus $-0.1\%$, $-0.6\%$, $-3.5\%$ for uniform 4-bit quantization). Its advantage grows with the horizon, reaching $-9.0\%$ (TimeMoE) and $-7.7\%$ (Moirai) at $H=720$.