SLiK: Learned Sturm–Liouville Spectral Geometry for Sequence Modeling

A central challenge in sequence modeling is to represent local structure and long-range dependencies within a unified architecture. Spectral methods provide compact global representations, but typically operate in predefined coordinate systems, such as Fourier or polynomial bases, learning only transformations within them and leaving the retained global subspace largely fixed. We ask whether the coordinate-defining operator can instead be learned from the task. We introduce SLiK, a local–global architecture that learns a continuous, self-adjoint Sturm–Liouville operator end-to-end. Its low-order eigenspace defines task-adapted global coordinates, within which eigenvalue-conditioned spectral responses transform sequence information, while a parallel multi-scale convolutional path preserves local and high-frequency structure. For a fixed number of retained modes, spectral projection and reconstruction scale linearly with sequence length, and the continuous operator can be re-discretized across sequence lengths without introducing length-specific learned parameters. Across four long-term forecasting benchmarks, SLiK is competitive with strong Transformer, Mamba, and MLP baselines. Under matched 180M-parameter, 100B-token language-model pretraining, SLiK remains competitive with strong Transformer and state-space baselines while achieving 80.6% average needle-in-a-haystack retrieval accuracy. On long-read RNA splicing classification, SLiK achieves 0.9063 accuracy with the fastest runtime among evaluated models. Controlled ablations against fixed spectral bases, generic learned orthogonal bases, and frozen operators, with closely matched parameter counts, show that structured operator learning provides gains beyond generic basis adaptation. Together, these results support learning spectral geometry itself as a general design principle for sequence modeling across modalities.