S${}^3$M: Spectral State Space Models for Structured Multivariate Time Series

Multivariate streams exhibit temporal memory and structured interactions among variables. Standard selective state space models capture the former but rely on channel-separable transitions, whereas graph neural networks capture the latter without a recurrent state-space interpretation. We propose the spectral state space model (S${}^3$M) for structured multivariate time series, a graph-structured state space model in which the latent state is a graph signal and the linear state transport is a learned mixture of graph-spectral filters. A dynamic observation graph captures context-dependent relations, a compact B-spline filter bank provides frequency-selective transport, and a top-s router selects a sparse spectral subspace at every time step. Availability-aware pooling prevents unobserved entries from receiving readout weight. We establish exact graph-Fourier diagonalization of the linear transport, characterize when it produces cross-variable mixing in measurement coordinates, prove optimality of the implemented top-$s$ simplex rule, and bound the state of the gated, layer-normalized recurrence. Evaluated on the large-scale benchmark PhysioNet 2019, our method outperforms the state-of-the-art (SOTA) baselines on the standard primary metric and shows effectiveness. Controlled interventions quantify the contributions of spectral filtering, recurrent memory, graph coupling, and availability-aware readout.