- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper investigates how heterogeneous time constants across units in a recurrent neural network impact computational performance. Even if time constants, 's, are set to be uniform the recurrent connectivity, generates a range of time constants. The authors show that one can amplify the diversity of timescales by setting the time constants to be non-uniform and "aligning" them with the timescales implied by . This might have computational benefits. A preliminary investigation of MEG data is undertaken to search for biological evidence of this "alignment".
Critical strengths
-
I like that the paper points out that the "benefits" of having heterogeneous time constants can be achieved by training directly. My sense is that previous papers have potentially oversold the benefits of baking in this heterogeneity. This paper does a decent
-
The authors show that early learning is faster when W_rec is trainable and the time constants are aligned to amplify fast and slow modes. They also show in a reservoir
Critical weaknesses
-
I found this paper extremely difficult to read and honestly I did not understand the motivation behind the project or its significance. The series of experiments are introduced in section 2 without much explanation and I found the composition of the paper disjointed—When reading section 3, I had to refer back to section 2 to understand what was going on.
-
The tasks studied in this paper are very simple, toy problems. I am not against studying toy problems, but they need to be better motivated. For an ML audience, there is no strong connection to genuinely challenging modern tasks. For a neuroscience audience, the mapping to biology isn't made clear. Furthermore, these tasks have been extensively studied by Yang et al. (2019) and Driscoll et al. (2024) and its not clear what new insights the paper here provides.
-
The analysis of the human MEG dataset is potentially a new link to biology, but it is hard to interpret the results. The authors themselves say it provides "supportive but limited evidence". If I understand it correctly, the authors only have access to the effective timescales measured by MEG. The effective timescales reflect a mixture of recurrent connectivity and intrinsic time constants for each region, and it is hard to disentangle the two with this dataset. Since the rest of the paper deals with the intrinsic timescales, there is not direct evidence for or against any of the key points in the paper.
Questions for authors
- Who is the target audience for this paper?
- What are the take home messages for that audience?
- What future directions of research are enabled by your contribution?
2 comments
Weakness 3 — interpretation of the human MEG analysis
We appreciate the reviewer's comment about the distinction between the measured timescales of cortical activity and the intrinsic time constants of individual neurons. MEG timescales reflect effective population dynamics shaped by intrinsic neuronal properties, recurrent interactions, and their combined effects. In an additional analysis, we used the empirical 400-region structural connectome as a recurrent model in order to investigate whether connectivity can generate the observed cortical timescale hierarchy. To this end, we set the intrinsic time constants of all regions to the same value (, ).
When we analyse the effective timescales generated by the recurrent model, we find a positive correlation with empirical MEG timescales (Spearman , spatial rotation ). This positive association suggests that recurrent connectivity may contribute to the empirical spatial ordering. Including the slow-mode participation in the recurrent model as an additional predictor increased the explained variance by (spatial rotation ). Importantly, assigning the empirical MEG timescale ordering to intrinsic time constants in the connectome model remains a modelling assumption. Our controlled recurrent network experiments demonstrate the computational consequences of intrinsic timescale placement, and the cortical analysis examines the association between measured activity timescales and slow recurrent organisation. The cortical findings therefore provide a complementary biological perspective on the role of slow recurrent organization.
Changes in the revised manuscript: Sec. 3.5, lines 466–474; Discussion, lines 522–531; Appendix H, lines 1150–1159.
We appreciate the reviewer's recognition that recurrent training can compensate for the benefits of intrinsic τ heterogeneity, and that aligned placement improves early learning. We also thank the reviewer for highlighting the need to improve the clarity of the manuscript's motivation, significance, and organization. These comments helped us clarify the central question and make the importance of our work more accessible to a broader audience. We have accordingly revised the Introduction and the presentation of our experiments.
Weakness 1 — motivation and presentation
We appreciate the reviewer's point on the narrative and clarity. The central question of our work is not simply whether heterogeneous intrinsic timescales are beneficial for recurrent computation; recurrent connectivity itself can generate a diverse set of effective timescales, even when the intrinsic time constants of the neurons are identical. Thus, the question we address is a more specific one in computational neuroscience, namely whether changing the assignment of the same set of heterogeneous intrinsic timescales across neurons in a network changes the network's dynamics and computation.
In our experiments, we first investigate the contribution of heterogeneity to computation with and without recurrence. We then isolate the effect of intrinsic timescale placement on effective dynamics and task performance, and investigate whether the initial placement affects learning when recurrent weights are trained. Finally, we investigate whether cortical activity timescales are associated with slow modes of the human structural connectome. We have clarified the motivating question in the Introduction and at the beginning of the relevant Results subsections to make the experimental progression easier to follow.
Changes in the revised manuscript: Introduction, lines 60–76; Results, lines 296–299, 344–345, 407–408, and 455–456.
Weakness 2 & Questions 1–3 — task complexity, significance, target audience, and future directions
We thank the reviewer for raising the questions of significance, intended audience, and future directions for our work.
Aim Instead of evaluating performance on complex ML tasks, our aim is to establish the computational role of intrinsic timescale placement with controlled recurrent network experiments.
Contribution We have shown that the contribution of intrinsic timescale heterogeneity to network computation depends not only on which time constants are present, but also on where they are located in the recurrent network. Alignment with slow recurrent modes improves early learning, while training recurrent weights partly compensates for unfavourable initial placement.
Audience Our work is intended for computational neuroscience researchers interested in studying neural timescales and recurrent dynamics, as well as machine learning researchers interested in studying temporal computation and learning in recurrent networks.
The tasks we use in our experiments are established paradigms in computational neuroscience designed to investigate principles of neural computation under controlled conditions. We studied the placement principle across four different types of memory and decision tasks, three different types of network topology, and network sizes of 64, 128, and 256 units. In addition, we clarified the contributions and scope of our findings in both the Introduction and Discussion. Possible future directions include extending the study to more complex temporal tasks and larger networks; studying learning effects beyond MemoryPro; and investigating how intrinsic neuronal properties interact with recurrent dynamics in biological circuits.
Changes in the revised manuscript: Introduction, lines 60–76 and 98–106; Discussion, lines 533–537.