- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper used recurrent neural networks (RNNs) to investigate the candidate computational mechanisms for sequential working memory (WM) tasks. This work is motivated by recent experimental findings showing that the near orthogonal subspaces were formed during sequential WM with multiple items. The authors trained networks to solve tasks that are analogous to the task paradigms used in the monkey experiments. While various previous studies had used RNNs to study similar tasks, the model developed here made several notable modifications, including the inclusion of inter-stimulus intervals and distinct encoding, maintenance and retrieval periods.
The authors performed comprehensive analysis on the RNNs trained on the basic vision of the task to understand how neural dynamics recruit different subspaces in different phases of the trial to solve the task. The procedures used are generally sound and the results are interesting. Finally, the authors tested how well the RNNs generalized within and across sequence lengths, and also used a more complex handwritten sequence reproduction task to study the RNNs reduce sensory-memory interference.
Overall, this is a solid submission with solid contributions. Despite the dense writing (due to that many results were presented), the paper is well written. The logic in the paper is mostly clear and relatively easy to follow. My main concern is that several recent papers have used similar approaches to study sequential WM, and there are some overlaps between the ideas presented here and those papers (in particular, Wang et al, ICLR workshop paper, 2025). Having said that, the current paper has some novel contributions.
Critical strengths
-
The analyses and results on the dynamics of RNNs are pretty comprehensive and impressive. The authors carefully studies the geometrical structure of the subspaces each phase of the tasks, and how these subspaces for different items relate to each other. The authors also conduct additional numerical experiments by training RNNs to solve related and more complex tasks to test the generality of the results. Collectively, these resulted in new understandings of how neural systems may solve sequential working memory tasks.
-
The tasks the RNNs were trained on better match the behavioral paradigms used in the monkey experiments. One aspect is the inclusion of inclusion of inter-stimulus intervals to decouple time with the stimulus rank. Notably, their models can explain some experimental observations that can not be explained by some previous models, for example, the dependence of the size of the stimulus encoding manifold on the rank.
Critical weaknesses
— Several previous papers have used similar paradigm to study WM. Some of the findings were also seen in prior work, such as the orthogonality of the subspaces. Some of the analyses reported here are similar to those in Wang et al (ICLR workshop, 2025).
- The concept of “schema” was not clearly define in the paper and remains unclear. The claim on the reusable schema in the RNNs seems an overclaim to me.
Questions for authors
Line 168, here the authors mentioned attractor dynamics. Any evidence for it?
Page 4, It s not clear to me how the PCA analysis in the regression weights related to PCA of neural activity. Given the factorized subspaces seen from one analysis, should we expect a factorized subspace from the other analysis? Are they dual?
Line 265-268, is the subspace defined by the leading PC pair a fixed subspace? Or was it changing over time? I am under the impression that it should be changing over time. If that is indeed the case, using the word “it” to refer to such subspaces may be confusing.
Line 280, what does low-similarity imply about the geometry of the stimulus encoding within these subspaces? It is not clear to me what this CCA analysis captures.
The authors highlighted the important of variable ISIs. If using fixed ISIs, would that indeed lead to different predictions?
What is the relationship of the results here and those in Driscoll et al (2024)? A brief discussion might be useful.
3 comments
Thank you for your response to my questions. The clarification and revision help make several points more clear. I will maintain my positive rating of the paper.
We thank the reviewer for carefully considering our responses and for the continued positive assessment of our work. The reviewer’s constructive feedback has helped us clarify key points and improve the manuscript.
We thank the reviewer for constructive suggestions and for recognizing our contributions, novelty, clear logic and comprehensive analyses of RNN dynamics.
W1: We agree that prior work reported near-orthogonal rank subspaces. We appreciate the reviewer’s recognition of mechanistic insights enabled by our temporally extended task. Our specific contributions are
W2: We agree that “schema” and “reusable” needed clarification. An ordinal schema is a learned task-specific organization of rank subspaces encoding item identities at corresponding ordinal positions. Reuse primarily means applying it to unseen combinations of familiar items within this task family without retraining: all 40 models correctly recalled their held-out sequences, and length-3 models generalized to lengths 1–2.
Q1: We thank the reviewer for asking about attractor dynamics. The local Jacobian projected onto the maintenance subspace had a mean spectral radius slightly above unity, near the marginal-stability boundary (Fig. 3I). We replaced attractor dynamics with recurrent dynamics.
Q2: Both analyses reveal factorized rank geometry in our RNN. In GRU/LSTM controls, coefficient projections resembled the RNN’s, but activity PCs showed more rank mixing, especially ranks 2–3 (Appendix C.8; Figs. S6–S7). Thus, both analyses are complementary, not necessarily mathematical duals. This difference may reflect gating or LSTM cell states support alternative memory dynamics, persistent RNN activity qualitatively parallels experimental results.
Q3: For each retrieval window, we fit PCA to activity averaged over time within each trial (Appendix B.3.2). Each basis is fixed for that window’s analysis, and its leading PC pair primarily represents the recalled item (Fig. S19A–C). We revised Sec. 2.2.
Q4: CCA finds maximally correlated linear combinations of two multivariate datasets. In Fig. 3A, higher similarity for matched than mismatched ranks therefore indicates that same-rank representations remain more closely related across stages. Since CCA is invariant to such invertible linear transformations, the high similarity for matched rank subspaces could remain high though they may undergo rotation and scaling across time.
Fig. S14 illustrates why similar ring shapes can still yield low CCA. Coloring all subspaces by one rank’s item identity produces same-color clusters in that rank’s subspace but dispersed points in the others. The overall shape is similar, but the item information is organized differently, helping explain the low similarity between mismatched ranks. See more in Appendix B.4.
Q5: We used variable ISIs to encourage timing-independent ordinal representations, consistent with Dehaene et al.’s (2015, Neuron) account of ordinal knowledge. Training with a fixed 8-step ISI preserved rank organization, but doubling the test ISI reduced sequence recall from 100% to 49.2±21.1% (5 seeds).
Q6: Both studies examine how RNNs reuse and reconfigure internal structure to support WM computatoin. Driscoll et al. (2024) identified shared dynamical motifs supporting single-item representations across tasks. Our task requires preserving identities and ordinal relationships across multi-item encoding, maintenance and retrieval. Building on SWM findings [1–2], we examine rank subspaces as a reusable template for ordinal processing, an abstract level of sequence organization (Dehaene et al., 2015). Our ordered multi-item focus complements their multitask perspective. We have added this discussion.
[1] Xie et al., Science, 2022
[2] Chen et al., Neuron, 2024