acceptodds
Under review as a conference paper at ICLR 2027

Learning and Dynamically Deploying Ordinal Schemas for Sequence Working Memory in RNNs

Abstract

Sequence working memory (SWM) requires neural systems to maintain item-specific information while organizing it according to ordinal structure over time. Experiments in the macaque prefrontal cortex have revealed a factorized geometry in which items at different ordinal ranks occupy low-dimensional, approximately orthogonal subspaces. However, how such geometry emerges in recurrent networks and, more importantly, how it is dynamically deployed to support computation across encoding, maintenance, and retrieval remain unclear. Here, we show that a recurrent neural network (RNN) develops a reusable ordinal schema comprising content-independent rank subspaces within which stimulus-specific representations are embedded. Crucially, network analysis reveals stage-dependent alignment of the currently relevant rank subspace with the fixed input- and output-weight spaces during encoding and retrieval, respectively. Meanwhile, neural dynamics within previously encoded subspaces continue to evolve in a structured manner, and activity remains marginally stable during maintenance, balancing memory retention with responsiveness to ongoing computational demands. The learned schema generalizes across tested sequence lengths, while perturbations in a richer stroke-based task remain largely confined to the affected rank, limiting cross-rank interference. These results suggest that rank-specific geometry provides a reusable computational scaffold through which recurrent circuits dynamically support flexible, generalizable sequence processing in both biological and artificial recurrent systems.

Primary area:
applications to neuroscience & cognitive science
Keywords:
sequence working memory, recurrent neural networks, schema

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_vb7HOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper used recurrent neural networks (RNNs) to investigate the candidate computational mechanisms for sequential working memory (WM) tasks. This work is motivated by recent experimental findings showing that the near orthogonal subspaces were formed during sequential WM with multiple items. The authors trained networks to solve tasks that are analogous to the task paradigms used in the monkey experiments. While various previous studies had used RNNs to study similar tasks, the model developed here made several notable modifications, including the inclusion of inter-stimulus intervals and distinct encoding, maintenance and retrieval periods.

The authors performed comprehensive analysis on the RNNs trained on the basic vision of the task to understand how neural dynamics recruit different subspaces in different phases of the trial to solve the task. The procedures used are generally sound and the results are interesting. Finally, the authors tested how well the RNNs generalized within and across sequence lengths, and also used a more complex handwritten sequence reproduction task to study the RNNs reduce sensory-memory interference.

Overall, this is a solid submission with solid contributions. Despite the dense writing (due to that many results were presented), the paper is well written. The logic in the paper is mostly clear and relatively easy to follow. My main concern is that several recent papers have used similar approaches to study sequential WM, and there are some overlaps between the ideas presented here and those papers (in particular, Wang et al, ICLR workshop paper, 2025). Having said that, the current paper has some novel contributions.

Critical strengths
  • The analyses and results on the dynamics of RNNs are pretty comprehensive and impressive. The authors carefully studies the geometrical structure of the subspaces each phase of the tasks, and how these subspaces for different items relate to each other. The authors also conduct additional numerical experiments by training RNNs to solve related and more complex tasks to test the generality of the results. Collectively, these resulted in new understandings of how neural systems may solve sequential working memory tasks.

  • The tasks the RNNs were trained on better match the behavioral paradigms used in the monkey experiments. One aspect is the inclusion of inclusion of inter-stimulus intervals to decouple time with the stimulus rank. Notably, their models can explain some experimental observations that can not be explained by some previous models, for example, the dependence of the size of the stimulus encoding manifold on the rank.

Critical weaknesses

— Several previous papers have used similar paradigm to study WM. Some of the findings were also seen in prior work, such as the orthogonality of the subspaces. Some of the analyses reported here are similar to those in Wang et al (ICLR workshop, 2025).

  • The concept of “schema” was not clearly define in the paper and remains unclear. The claim on the reusable schema in the RNNs seems an overclaim to me.
Questions for authors

Line 168, here the authors mentioned attractor dynamics. Any evidence for it?

Page 4, It s not clear to me how the PCA analysis in the regression weights related to PCA of neural activity. Given the factorized subspaces seen from one analysis, should we expect a factorized subspace from the other analysis? Are they dual?

Line 265-268, is the subspace defined by the leading PC pair a fixed subspace? Or was it changing over time? I am under the impression that it should be changing over time. If that is indeed the case, using the word “it” to refer to such subspaces may be confusing.

Line 280, what does low-similarity imply about the geometry of the stimulus encoding within these subspaces? It is not clear to me what this CCA analysis captures.

The authors highlighted the important of variable ISIs. If using fixed ISIs, would that indeed lead to different predictions?

What is the relationship of the results here and those in Driscoll et al (2024)? A brief discussion might be useful.

3 comments

We thank the reviewer for constructive suggestions and for recognizing our contributions, novelty, clear logic and comprehensive analyses of RNN dynamics.

W1: We agree that prior work reported near-orthogonal rank subspaces. We appreciate the reviewer’s recognition of mechanistic insights enabled by our temporally extended task. Our specific contributions are

  1. Distributed rank coding across task stages. Regression-based analyses fit activity to item and rank variables before applying PCA to the regression coefficients. Instead, we apply unsupervised PCA directly to neural activity, using task labels only to interpret the projections, and the resulting rank-specific organization across stages resembles experimental distributed rank coding [1–2], contrasting with prior modular interpretations. Our model also reproduces experimental single-neuron tuning (Figs. S9-S10) and a decreasing manifold radius with rank (Fig. 2A), not reported in prior SWM models.
  2. A clear mechanistic link between input/output alignment and subspace routing. Same-rank representations remain linearly related across stages (Fig. 3A). During encoding, subspaces reorient relative to fixed input weights, supporting item routing while preserving earlier item–rank bindings (Fig. 3B–H). Recurrent dynamics maintain these bindings during the delay; successive alignment with fixed output weights supports sequential retrieval (Fig. S19D).
  3. Functional advantage of orthogonal organization. Perturbations in our handwriting task mainly affect the targeted rank, limiting cross-rank interference, consistent with the independent-failure prediction in [1].

W2: We agree that “schema” and “reusable” needed clarification. An ordinal schema is a learned task-specific organization of rank subspaces encoding item identities at corresponding ordinal positions. Reuse primarily means applying it to unseen combinations of familiar items within this task family without retraining: all 40 models correctly recalled their held-out sequences, and length-3 models generalized to lengths 1–2.

Q1: We thank the reviewer for asking about attractor dynamics. The local Jacobian projected onto the maintenance subspace had a mean spectral radius slightly above unity, near the marginal-stability boundary (Fig. 3I). We replaced attractor dynamics with recurrent dynamics.

Q2: Both analyses reveal factorized rank geometry in our RNN. In GRU/LSTM controls, coefficient projections resembled the RNN’s, but activity PCs showed more rank mixing, especially ranks 2–3 (Appendix C.8; Figs. S6–S7). Thus, both analyses are complementary, not necessarily mathematical duals. This difference may reflect gating or LSTM cell states support alternative memory dynamics, persistent RNN activity qualitatively parallels experimental results.

Q3: For each retrieval window, we fit PCA to activity averaged over time within each trial (Appendix B.3.2). Each basis is fixed for that window’s analysis, and its leading PC pair primarily represents the recalled item (Fig. S19A–C). We revised Sec. 2.2.

Q4: CCA finds maximally correlated linear combinations of two multivariate datasets. In Fig. 3A, higher similarity for matched than mismatched ranks therefore indicates that same-rank representations remain more closely related across stages. Since CCA is invariant to such invertible linear transformations, the high similarity for matched rank subspaces could remain high though they may undergo rotation and scaling across time.

Fig. S14 illustrates why similar ring shapes can still yield low CCA. Coloring all subspaces by one rank’s item identity produces same-color clusters in that rank’s subspace but dispersed points in the others. The overall shape is similar, but the item information is organized differently, helping explain the low similarity between mismatched ranks. See more in Appendix B.4.

Q5: We used variable ISIs to encourage timing-independent ordinal representations, consistent with Dehaene et al.’s (2015, Neuron) account of ordinal knowledge. Training with a fixed 8-step ISI preserved rank organization, but doubling the test ISI reduced sequence recall from 100% to 49.2±21.1% (5 seeds).

Q6: Both studies examine how RNNs reuse and reconfigure internal structure to support WM computatoin. Driscoll et al. (2024) identified shared dynamical motifs supporting single-item representations across tasks. Our task requires preserving identities and ordinal relationships across multi-item encoding, maintenance and retrieval. Building on SWM findings [1–2], we examine rank subspaces as a reusable template for ordinal processing, an abstract level of sequence organization (Dehaene et al., 2015). Our ordered multi-item focus complements their multitask perspective. We have added this discussion.

[1] Xie et al., Science, 2022

[2] Chen et al., Neuron, 2024

Thank you for your response to my questions. The clarification and revision help make several points more clear. I will maintain my positive rating of the paper.

We thank the reviewer for carefully considering our responses and for the continued positive assessment of our work. The reviewer’s constructive feedback has helped us clarify key points and improve the manuscript.

Reviewer_h1Q4OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper trains RNN models on a working memory task of random length-three sequences and analyzes the learned representation geometry and dynamics. The models replicate neural data from previous experiments well, including separate low-dimensional subspaces for different ranks in the sequence. They analyze in detail the dynamics of these subspaces and find that the subspaces are reused to some extent across task stages, from encoding to maintenance to retrieval, but also change over time to some degree so that the relevant rank subspace is aligned to the input-weight or output-weight subspace at the appropriate time. Subspaces generalize across sequence lengths. Perturbation experiments provide evidence that the subspaces operate relatively independently of each other, and results generalize to a slightly more complex set of sequences.

Critical strengths
  1. The task setup uses a more realistic, flexible timing compared to previous neural networks on sequence working memory, so the presentation of each stimulus in the sequence is followed by a delay of a variable amount of time, and there is a variable maintenance period between the stimulus encoding and retrieval periods. However, see weakness for my concern on the actual implementation.

  2. The flexible task timing leads to more interesting dynamics, and in particular the finding that subspaces change over time so the relevant subspace is aligned to the input weights or output weights at the appropriate time. However, the subspaces do not completely switch over time, so the subspace corresponding to the same rank maintains some similarity over time. This seems to be a novel mechanism that sits between the complete switches in Whittington et al., (2025) (where each item is passed down a series of subspaces during sequence presentation, and passed back up during retrieval; one can interpret this as the rank-ii-encoding subspace changing over time so that the correct subspace is aligned with the input weights or output weights), and the completely stationary subspaces in Wang et al., (2025), though no mechanism is provided for how such compromise is achieved (and see weakness).

Critical weaknesses
  1. The durations used in the task appear to be very short compared to the neuron time constant. The neuron time constant is tau=1, while each time step is only dt=0.05. If I understood correctly, the authors use an ISI of max 8 time steps, and maintenance period of max 30 time steps (this is less than 2 neuron time constants), so the network doesn’t really need to develop a lot of recurrent dynamics to actively remember the input. This seems to differ significantly from biological reality: for example, the biological neuron time constant is around 20ms, and the delay period of a working memory experiment can be on the scale of seconds, so the brain really has to fight against the intrinsic decay of input-driven activity and potentially use attractor-like dynamics to maintain information. A consequence of such fast trial timing is that it may exaggerate changes in the subspaces – for example, an attractor-based subspace could be quite stable over long time horizons, but it will need some time steps initially to stabilize and reach the fixed point first, with very fast trials it might only catch this initial period
  2. There seems to be some discrepancy between the CCA results in Fig. 3A, which suggest that the same rank subspace is quite stable across task stages, compared to Fig. 3B-C and Fig. 3F-H, which suggests that the same rank subspace is changing quite a bit over time during stimulus presentation in order to be aligned to input weights at a specific stage. For example, S1-U1 subspace and S2-U1 subspace have quite high CCA similarity (judging by eye on the color scale, maybe around 0.8) in Fig. 3A, but their angle is around 50 degrees, and S1-U1 is much more aligned to input weights compared to S2-U1. I have a conceptual confusion related to this: Fig. S22 suggests that these models have a separate sensory subspace (similar to neural data) that encodes whatever stimulus is currently being presented. If that’s the case, wouldn’t one expect that the S1-U1, S2-U2, and S3-U3 subspaces to basically be this sensory subspace, which is different from the later U1/U2/U3 subspaces during maintenance? This would explain the increased alignment to input weights, however Fig. 3A seems to show that there is not a lot of similarity among S1-U1, S2-U2, and S3-U3; and instead there is high similarity between e.g. S1-U1 and maintenance M-U1.
Questions for authors
  1. Can networks trained on these short trials generalize to much longer trials? If so, then my concern on the short trial durations may be resolved
  2. Which specific time steps were used to estimate the subspaces during stimulus presentation (is it during early stimulus onset, or during the ISI after)? That may change the interpretation of the subspace dynamics.
5 comments

We thank the reviewer for the constructive suggestions and positive assessment.

W1/Q1: We thank the reviewer for raising the important question and agree that the original trial durations were short relative to the delay duration in SWM.

In additional experiments, we progressively extended the delay to 1 s while gradually reducing the learning rate from 10⁻³ to 5×10⁻⁵ (Appendix C.9). Taking τ=20 ms and Δt/τ=0.05, 1 s corresponds to 1,000 steps (50τ), approximately 33–50 times the original 20–30-step delay. The models achieved 100% accuracy at 1 s (5 seeds). Without further training, accuracy remained 97.17±2.17% (mean±SD, 5 seeds) at 10 s (500τ). See Table S5 for performance at 2 s, 3 s, and 5 s.

The neural representations were also retained. Items formed ring-like manifold within nearly orthogonal rank subspaces during maintenance, while successive inputs recruited additional PCs during encoding (Fig. S32). CCA similarity patterns across encoding, maintenance and retrieval closely matched those of the original models (Pearson r=0.919). At 1 s, late-maintenance activity resembled that of the original models under two complementary measures: linear CKA, which compares population activity structure allowing rotation and uniform scaling, was 0.972; RSA, which compares relative distances between sequence representations, was 0.964. Similarity remained high even at 10 s (Table S5).

Stable memory over longer delays, with similar rank geometry and cross-stage relationships, supports the original timing's adequacy for revealing the core representational organization.

W2: We thank the reviewer for raising this question. CCA finds maximally correlated linear combinations of two multivariate datasets. In Fig. 3A, we measure linear correspondence between activity projected onto rank subspaces across stages and find higher similarity between matched rank subspaces, such as S1–U1 and S2–U1, than between mismatched ones. High CCA is compatible with their ~50° angle (Fig. 3B), since CCA is invariant to invertible linear transformations, including rotation, scaling and shear.

We apologize for the unclear description of the shared sensory subspace. Following Chen et al. (2024), we construct this subspace using regression-based PCA. Linear regression estimates each neuron’s stimulus-identity coefficients at each ordinal rank. For each stimulus-presentation period, we select the coefficients corresponding to the currently presented rank and average these coefficient matrices across periods. Applying PCA to the averaged matrix yields a common sensory projection space across ranks (Appendix B.2.2). However, rank subspaces in Fig. 3A are instead identified by applying PCA directly to population activity averaged over time within the task window (see our response to vb7H, W1). Thus, S1–U1, S2–U2 and S3–U3 in Fig. 3A have different operational definitions from the shared sensory subspace and should not be equated with it. We use these window-specific activity-PC rank subspaces to quantify their alignment with fixed input/output spaces and their reorientation across task stages.

S2: We thank the reviewer for recognizing our flexible task timing reveals dynamic subspace alignment with input/output weights. As the reviewer notes, our rank subspaces reconfigure while retaining cross-stage correspondence, suggesting a potential novel intermediate mechanism between the complete switching described in [1] and the stationary subspaces reported in [2].

A possible explanation for complete switching in [1] is their use of shared input/output interface. In their example (Fig. 1A, D–E), items enter a common slot that also serves as readout. Recurrent dynamics transfer representations between slots to bring the required item to this interface. Within this scheme, accessing different items thus requires switching between subspaces.

By contrast, the fixed-slot solution in [2] uses learned sensory, memory and readout modules. Recurrent connections maintain each item's content within its slot. Inhibitory gating controls successive inputs, while chain of excitation activates the readout populations. Connectivity and storage subspaces remain fixed, but pathway activation changes over time, enabling sequential access without shifting contents between memory slots.

Q2: We thank the reviewer for raising this question. In Fig. 3A–C and F–H, PCA was fitted separately for each stimulus presentation using activity averaged across all its timesteps within each trial, excluding the subsequent ISI. Additional analyses restricted to late stimulus presentation or the subsequent ISI yielded the same qualitative pattern: the most recently presented item’s subspace aligned more closely with the input-weight subspace, while earlier items’ subspaces reoriented over time (Appendix C.12; Table S6). We have clarified these windows in the revised Methods.

[1] Whittington et al., 2025

[2] Wang et al., 2025

Thank you for the detailed responses and substantial additional experiments.

The additional models trained with longer delays considerably strengthen the paper and largely address my original concern about the short maintenance periods. My main remaining concern is the interpretation of the dynamic alignment of rank subspaces with the input and output weights. I agree that Fig. 3C and Table S6 provide evidence that the currently encoded item's subspace is preferentially aligned with the input weights. However, (1) the ISIs are still extremely short (for comparison, experiments in Xie et al. (2022) and Chen et al. (2024) use ISIs of around 400 ms, corresponding to 400 time steps in the current simulations), and relatedly, (2) it remains unclear whether the memory-storage subspaces themselves change over time, or whether the observed changes in the PC subspaces mainly reflect the decreasing contribution of a transient sensory representation. Fig. S24 shows that the shared sensory subspace transiently represents each item as it is presented, which could contribute to the observed reorientation as sensory activity gives way to memory activity. It would be very informative to examine memory-specific rank subspaces over time, ideally with longer ISIs, to determine whether the representations of previously encoded items genuinely reorient as subsequent stimuli arrive, or whether the observed changes primarily reflect changing contributions of the shared sensory and memory representations. It would also strengthen the paper to demonstrate that the findings on subspace reorientation and input-weight alignment (Fig. 3B–C and Table S6) persist in the new networks trained with longer delays, in addition to the maintenance-period geometry and CCA patterns.

Thank you for clarifying the CCA analysis. This resolves my original confusion about the apparent discrepancy between Fig. 3A and Fig. 3B–C, but it also makes me question what Fig. 3A tells us about the geometry of the subspaces. CCA is applied to the trial-level PC coordinates, rather than the PC loading vectors in neural space, so it does not directly compare the actual neural subspaces. Low CCA between different ranks is largely expected, as they encode different item identities. Similarly, high CCA between the same rank across different stages indicates that the item information is preserved in linearly related representations, but does not imply that the same neural directions are reused. For example, two representations in completely orthogonal neural subspaces could still have perfect CCA if their item codes are linearly related. Thus, it seems that the block-diagonal CCA pattern in Fig. 3A could arise in both a fixed-slot model and a shifting-slot model, such as those in [1] and [2], despite their very different underlying dynamics. It also does not tell us whether, for example, the subspace representing the second item during S2 overlaps with the subspace representing the first item during S1, as one might expect from a shared sensory subspace. I therefore think Fig. 3A provides limited evidence for the actual reuse of rank subspaces across stages. I would appreciate some clarification on this distinction and how the results support the proposed mechanism relative to previous models.

We thank the reviewer for the constructive follow-up and for recognizing that the longer-delay experiments strengthen the paper.

1. Subspace reorientation with longer ISIs and delays.

We thank the reviewer for suggesting that we examine subspace dynamics with longer ISIs. We additionally trained models with ISIs extended to 100 ms and maintenance delays up to 1 s. With τ=20 ms, the ISI spans 5τ, compared with the original 8 ms ISI. All five seeds achieved perfect accuracy. The reorientation and input-alignment results also persisted with longer ISIs and delays, extending our findings to longer timing (new Fig. S34, Table S7).

We measured the reorientation of activity-PC-defined rank subspaces relative to the corresponding subspaces fitted during the preceding stimulus presentation in our newly added Fig. S34. We found that activity-PC subspaces continued to reorient after stimulus offset:

(1) During the first ISI, S1–U1 continued to reorient for approximately 60 ms after S1 offset, possibly reflecting the influence of the S1-evoked activity trace. Its orientation relative to the stimulus-period subspace then stabilized for the rest of the ISI (Fig. S34A).

(2) During the second ISI, S2–U2 similarly continued to reorient for approximately 60 ms after S2 offset, after which its orientation stabilized. S2–U1 continued to reorient throughout the ISI (Fig. S34B).

(3) After all three stimuli had been presented, all three rank subspaces continued to reorient during approximately the first 200 ms following S3 offset, and then stabilized for the rest of the delay (Fig. S34C).

These dynamics are consistent with a contribution from sensory-driven transients, as the reviewer suggests, while demonstrating continued reorientation after item input.

We would also like to note that Fig. S24 uses regression-defined sensory and memory subspaces (Appendix B.2.2; our first response, W2). Here, rank subspaces are identified directly from population activity, thus is different from regression-based subspaces.

2. Slot organization revealed by activity PCA and cross-stage CCA.

We thank the reviewer for the constructive feedback and for acknowledging our clarification of the CCA analysis. We would like to further clarify the role of CCA in our analysis.

Here, we use direct activity PCA to identify rank-specific item representations within each stage, and CCA to test whether their content of items remain linearly related across stages. Our analysis addresses whether slots can be identified directly in the network's neural activity space. As described in Sec. 2.2, PCA is applied to population activity without using rank or item labels to construct the subspaces. The resulting PC1–2, PC3–4 and PC5–6 subspaces respectively represent item content at ranks 1–3 during maintenance, with these representations appearing successively during encoding (Fig. 2B–E). Within each subspace, item identities form distinct, orderly clusters. These results reveal a concrete neural realization of slots: subspaces in the dominant population activity that selectively carry item content at particular ordinal positions.

Identifying these representations separately within each stage, however, leaves open how their content of items correspond across stages. Separate visualizations alone do not provide a quantitative comparison of the item information represented in different subspaces. Even when these subspaces encode the same item identities, their codes need not be linearly related. High CCA for matched ranks and low CCA for mismatched ranks in Fig. 3A support a rank-specific linear correspondence across encoding, maintenance and retrieval. Here, cross-stage reuse and shared ordinal geometry in Sec. 3 refer to preserved rank–content organization, while the underlying neural directions may change.

In [1,2], each regression-based rank subspace is obtained by applying PCA to regression coefficients estimated with the relevant rank and item identities specified in advance. Given the already established item geometries, similar CCA pattern would be expected in [1,2] naturally. Nevertheless, our activity-PC subspaces are instead fitted directly and unsupervisedly to the full population activity, with labels used afterwards to interpret the projections. CCA then tests the cross-stage linear correspondence of these independently identified item codes. This is an additional empirical result rather than guaranteed by identifying rank-selective representations at each stage.

[1] Whittington et al., 2025

[2] Wang et al., 2025

Reviewer_PkiBOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Summary of contributions:

  • The paper develops a computational account of the ordinal organization observed in macaque prefrontal cortex during sequence working memory.
  • The authors train an RNN on a delayed serial-recall task. Three stimuli are presented sequentially, followed by a maintenance period and a recall phase in which the network reproduces the stimulus identities in their original order.
  • The trained RNN develops low-dimensional, approximately orthogonal rank-specific subspaces that separate ordinal position from stimulus content, resembling previously reported neural signatures. The authors analyze how these representations are dynamically configured across encoding, maintenance, and retrieval. A separate held-out experiment demonstrates recall of previously unseen combinations of familiar items.
Critical strengths

Strengths:

  • Novelty and significance: I found this paper particularly interesting. The emergence of an interpretable representation that separates ordinal rank from stimulus content provides a useful connection between task-trained recurrent networks and findings from macaque prefrontal cortex. The analyses offer insight into how a reusable representation can be dynamically accessed for encoding and retrieval, extending the contribution beyond reproducing a static geometric signature.
  • Temporally extended task structure: The task meaningfully extends simplified sequence-memory paradigms, including those considered in prior work such as Whittington et al. (2025). Multi-timestep stimulus presentations, randomized inter-stimulus intervals, an explicit maintenance period, and temporally extended serial recall allow encoding, storage, and retrieval to be examined separately. Although the task remains relatively simple, this structure makes the mechanistic analysis more informative.
Critical weaknesses

Weaknesses:

  • Robustness to stimulus and network dimensionality: The input/output coding controls are useful, but the principal findings rely on a 400-unit RNN processing sequences drawn from six stimulus identities. A more systematic investigation of recurrent width and stimulus-set size would help establish how the ordinal geometry depends on the relationship between network capacity and task dimensionality. It would also be informative to examine whether the organization persists under sparse connectivity or activity constraints. These experiments would strengthen the generality of the proposed mechanism.
  • Clarity of the task description: My main presentation concern is that understanding the task and its input/output structure requires effort for someone not working particularly in this area. As a reader less familiar with these paradigms, I initially found it difficult to determine what the network receives and what it must output at each stage. Bringing an explicit input–target timeline, such as Figure S1, into the main text would help. The description should clearly distinguish the 24 population-coded stimulus channels, six stimulus identities, fixation signal, and per-timestep outputs. Spatial locations or identity labels should accompany the colors, with an explicit statement that color is used only for visualization.
  • Neural predictivity: The correspondence with previously observed neural geometry is interesting, but a direct neural predictivity analysis would strengthen the biological interpretation. A useful extension would fit a mapping from model activity to recorded PFC responses under matched task conditions and evaluate predictions on held-out trials or conditions. Comparing against alternative models with similar behavioral performance would help determine whether the learned ordinal organization provides additional explanatory value for neural responses.
Questions for authors

Please refer to my weaknesses section. I'm particularly curious how the ordinal structure relates to activation / sparsity constraints placed on the RNN.

1 comment

We thank the reviewer for the thoughtful and encouraging assessment of our work. We particularly appreciate the recognition that our analyses extend beyond reproducing a static geometric signature by examining how reusable ordinal representations are dynamically deployed to support sequence memory. We are also grateful for the positive assessment of our temporally extended task design, which allows encoding, maintenance, and retrieval to be examined separately and supports this mechanistic investigation.

W1: We thank the reviewer for this helpful suggestion. We assessed the robustness of ordinal geometry by separately varying recurrent width, stimulus-set size, recurrent connectivity, and activity regularization against a matched dense reference (400 recurrent units, six stimulus identities). Each configuration used five independent training seeds and three-item sequences (Appendix C.4). With six identities, 100- or 800-unit networks qualitatively preserved rank-specific organization during encoding and maintenance and approximate orthogonality between rank subspaces (Figs. S25–S26; Appendix B.3). 400-unit networks trained on four or twelve identities likewise retained these features (Figs. S27–S28).

We also tested recurrent sparsity and activity L1 regularization with 400 units and six identities. Sparse networks used a fixed mask retaining 20% of recurrent connections; regularized networks used dense connectivity with an L1 penalty on mean absolute recurrent activity. Both conditions preserved rank-related organization, and all models achieved 100% exact-sequence recall (Figs. S29–S30).

All six configurations also preserved the broad rank-related CCA similarity patterns across encoding, maintenance, and retrieval (Fig. S31; Appendix B.4). These results (revised Appendix C.4) support the robustness of ordinal organization across the tested widths, stimulus-set sizes, connectivity constraints and regularization settings.

W2: We thank the reviewer for highlighting this presentation issue. We added an explicit input–target timeline adapted from Fig. S1 to the main text (Fig. 1C), illustrating encoding, maintenance, and retrieval. We revised the description to distinguish the 24 population-coded stimulus channels, six stimulus identities, fixation signal, and per-timestep target outputs, and added explicit stimulus labels alongside the colors, which are used only for visualization.

W3: We thank the reviewer for raising this question. At the population level, the model reproduces approximately orthogonal rank subspaces, ring-like item organization, and progressively smaller representational radii at later ranks (Fig. 2), consistent with the findings of Xie et al. (2022). It also reproduces transient stimulus information in a shared sensory subspace followed by persistent information in rank-specific memory subspaces (Appendix C.10, Fig. S24), as reported by Chen et al. (2024).

This correspondence extends to individual neurons. Some model neurons preserve stimulus preferences across ranks while changing response amplitudes; others shift their preferred locations (Appendix C.11, Fig. S9), consistent with the experimental tuning patterns reported by Xie et al. and Li et al. (2025). Using the experimental definition and classification threshold for neuron-to-subspace strength also reveals a broadly similar tendency toward disjoint contributions across rank subspaces (Fig. S10), as reported by Li et al.

We agree that comparing predictions of PFC responses on held-out trials or conditions across behaviorally matched models would help test the explanatory value of ordinal organization. Our GRU and LSTM controls show why this comparison matters: perfect recall can coexist with different cross-stage representations (Appendix C.8; Fig. S8).

Our model suggests experimentally testable hypotheses. For example, based on our retrieval analysis (Fig. S19D), rank-specific representations should sequentially align with a common output-related subspace during serial recall. This could be tested by estimating that subspace from independent trials, holding it fixed, and tracking which rank’s representation aligns most closely with it during retrieval. We could use the data in Xie and Chen to further test our model’s predictions when the data from these two studies become publicly available.

Compared with the rank-separated population geometry emphasized in previous SWM models, the model–experiment correspondence demonstrated here is broader in scope. Within a single recurrent network, we jointly capture approximately orthogonal rank subspaces, the rank-dependent decrease in representational radius, sensory-to-memory transformations, and experimentally reported patterns of single-neuron tuning and subspace contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.