Select, Don't Compress: Sparse Attention for High-Dimensional Time Series Forecasting

In high-dimensional multivariate time series, forecasting accuracy may depend on specific interactions between individual channels. Dense cross-channel attention captures them, but its cost grows quadratically with the number of channels. Traditional alternatives either compress these interactions or remove them altogether, potentially losing the fine-grained dependencies that matter. We introduce Native Sparse Time Attention (NSTA), which preserves direct interactions between individual channels while scaling linearly in their number for a fixed sparsity budget. On both evaluated high-dimensional datasets, NSTA shifts the accuracy–efficiency Pareto frontier toward lower forecasting error.