Learning to Remember with Möbius Updates

Recurrent sequence models with matrix-valued hidden states provide fixed-size associative memory, but their update rules affect what information is kept. We study a family of linear-fractional memory updates that keep the state bounded in spectral norm, including affine updates and nonlinear updates induced by matrix Riccati dynamics. We give sufficient conditions for boundedness, analyze memory writing and retention, and derive a chunkwise form exactly equivalent to the sequential recurrence. Experiments on synthetic associative recall and state-tracking tasks examine accumulation, overwriting, and deletion, as well as generalization to longer sequences and increased memory load. Results show improvements over recurrent baselines in selected settings, with the relative performance of affine and nonlinear variants depending on the task and parameterization.