Learning and Dynamically Deploying Ordinal Schemas for Sequence Working Memory in RNNs

Sequence working memory (SWM) requires neural systems to maintain item-specific information while organizing it according to ordinal structure over time. Experiments in the macaque prefrontal cortex have revealed a factorized geometry in which items at different ordinal ranks occupy low-dimensional, approximately orthogonal subspaces. However, how such geometry emerges in recurrent networks and, more importantly, how it is dynamically deployed to support computation across encoding, maintenance, and retrieval remain unclear. Here, we show that a recurrent neural network (RNN) develops a reusable ordinal schema comprising content-independent rank subspaces within which stimulus-specific representations are embedded. Crucially, network analysis reveals stage-dependent alignment of the currently relevant rank subspace with the fixed input- and output-weight spaces during encoding and retrieval, respectively. Meanwhile, neural dynamics within previously encoded subspaces continue to evolve in a structured manner, and activity remains marginally stable during maintenance, balancing memory retention with responsiveness to ongoing computational demands. The learned schema generalizes across tested sequence lengths, while perturbations in a richer stroke-based task remain largely confined to the affected rank, limiting cross-rank interference. These results suggest that rank-specific geometry provides a reusable computational scaffold through which recurrent circuits dynamically support flexible, generalizable sequence processing in both biological and artificial recurrent systems.