Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence

As new evidence arrives, a sequence model must update both what it remembers and how that memory influences subsequent predictions. While Transformers incur computation and cache costs that scale with context length, fixed-state recurrent models offer constant-memory inference. However, linear and spectral recurrences traditionally rely on static, time-invariant transitions: fixed dynamics can append new content, but cannot dynamically revise how stored representations decay, rotate, or evolve. Although recent selective architectures introduce input-dependent transitions, they typically assign independent controls to every memory mode, coupling control cost directly to state capacity. We show that high-dimensional spectral memory does not require high-dimensional control, and introduce Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence (SPARC). SPARC employs just two input-dependent scalar signals to coordinate memory retention and phase rotation across heterogeneous complex modes, while preserving mode-specific baseline timescales and frequencies. Its diagonal, affine recurrence supports parallel associative scans for sequence-level BPTT, as well as exact structured Real-Time Recurrent Learning (RTRL) for online credit assignment. Across partially observable continuous control, POPGym, and sequence classification, SPARC achieves a $9.09\%$ relative return improvement on Walker-P and a $1.36\%$ relative accuracy gain on FordA over second-best methods. On an NVIDIA Blackwell GPU, our implementation reduces recurrent-mixer training latency by $18.2\%$–$34.2\%$ in fixed-token workloads and accelerates scans by $3.1$–$4.7\times$ over an optimized RG-LRU baseline. These results show that two shared control signals can efficiently govern adaptive spectral memory across both online and full-sequence learning settings.