acceptodds
Under review as a conference paper at ICLR 2027

Oscillatory Phase Coding Emerges in Adaptive RNNs for Flexible Working Memory

Abstract

Oscillatory phase coding (OPC) is widely observed in neural systems and has been implicated in cognitive functions such as memory and sequence processing. However, its computational role and neural substrates remain largely unclear. Here, we studied OPC by training recurrent neural networks (RNNs) with adaptation to perform sequence working memory tasks with variable delays. After training, we found that OPC emerges in the RNN spontaneously: the memorized items recur at distinct phases of a shared rhythm, with oscillation amplitude encoding item identity and phase encoding serial order. Ablation studies identified three key elements required for OPC: adaptation in neural dynamics, variable delay in memory retention, and flexibility in reading out information. Neuron-type analysis revealed the emergence of content and time cells in the trained RNN, and they process item identity and sequence order separately. Recurrent connectivity analysis revealed phase-organized connections among time cells and structured connections within digit-selective content-cell groups. Inspired by these findings, we proposed a neural circuit model which reproduces OPC through interactions between content units and an adaptive time ring. Together, this study reveals how OPC arises from network structures and task demands, offers insights into the implementation and computational mechanisms of OPC in biological systems, and inspires us to develop computational models for human-like cognitive functions.

Primary area:
applications to neuroscience & cognitive science
Keywords:
oscillatory phase coding, working memory, AI for neuroscience

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yod7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This article trains an adaptive RNN on a variable-delay sequence memory task. Oscillatory phase coding occurs without external rhythm or phase supervision: amplitude represents item content, while phase represents sequence order. The author identified the content and time cells and proposed a manually designed circuit to reproduce this organization.

Critical strengths

1.The phase coding that appears in task training is an interesting discovery.

2.Ablation links neural adaptation, delay variability, and readout timing with learned representations.

3.The separation of content and time provides a concrete and testable hypothesis.

Critical weaknesses

1.The main paper is not self-contained enough. The dataset source, train/test split, delay range, network size, and definitions of key analysis metrics are missing or deferred to the appendix. These details are necessary for evaluating the core claims and must be included in the main text.

2.The claim of necessity exceeds the evidence, and the failure to obtain stable phase coding in the ablation test does not necessarily mean that adaptation, variable delay, and flexible readout are usually necessary. The main paper still lacks aggregate results and uncertainty across independently trained networks.

3.The correlation between phase consistency and retrieval accuracy was reported, but it cannot be determined whether the network relies on phase coding to achieve successful recall. Targeted perturbations of phase relationships or identified cell populations will provide stronger evidence.

4.The circuit model builds in most of the proposed explanations, and its prescribed coupling gates and serial-order phases demonstrate a possible implementation, but it does not determine how these mechanisms are generated through learning or operate in biological systems.

Questions for authors

1.Can the main paper report on the basic experimental setup, metric definitions, accuracy, and phase consistency results across seeds?

2.Will disrupting phase relationships impair recall while retaining item representations?

3.Will a non-adaptive control with state and parameter matching yield the same conclusion about adaptation?

4.How does the proposed coding scheme handle repeated digits and different sequence lengths?

4 comments
AuthorsOpenReview ↗

Response to Reviewer yod7 (Part I/III)

We thank the reviewer for finding the emergent phase coding "an interesting discovery", the ablations informative, and the content/time separation "a concrete and testable hypothesis". Several concerns (notably W2 on multi-seed validation and W3 on causal interventions) are already addressed in the submission; below we point to their locations, clarify our presentation, and report the requested new experiments directly.

W1/Q1. Self-containedness of the main text

Within the strict page limit, our initial main text focused on the core scientific findings while documenting full implementation details and metric definitions in Appendices A and C, following standard ICLR practice (e.g., Dosovitskiy et al., 2021; Keller et al., 2024). Using the additional rebuttal page, we have added a dedicated "Experimental setup and metrics" paragraph directly to Section 2.2 of the main text (and clarified the definition of RϕR_\phi in Section 3.2) so that the main text is self-contained:

  • Data: MNIST digits; training images are drawn from the official training split, while validation and evaluation use separately sampled trials from the official test split.
  • Task: N=4N = 4 distinct digits, each presented for 1010 steps over two passes (T1=80T_1 = 80); variable delay T2∼U[50,150]T_2 \sim \mathcal{U}[50, 150]; retrieval window T3=55T_3 = 55.
  • Model: A-RNN with H=500H = 500 hidden units; test recall accuracy is 97.9%97.9\%.
  • Metrics: Oscillation amplitude and phase (sinusoidal fit to each digit logit), the phase-consistency index RϕR_\phi (20th percentile across one-cycle windows of the mean pairwise amplitude-weighted phase locking across trials), and retrieval accuracy.

W2/Q1. Multi-seed replications and scope of the necessity claim

Results across independently trained networks are already reported in the submission and cited in the main text (Section 3.1: "Appendix H.2; Figure 14"; Section 3.3: "Appendix H.3" and "Appendix H.4; Figure 16"). Specifically, Appendix E (Table 3; see also Table 4 for T2=220T_2 = 220) reports each ablation across 4–64\text{–}6 independently initialized networks (mean ± sample SD at T2=100T_2 = 100), which we summarize below alongside our new state-matched controls (Appendix J.1, Fig. 18a–g):

ConditionNetworks (nn)Test accuracy (%)Robust RϕR_\phi
Main A-RNN (variable delay, learned gate)1 (+12 configs below)97.810.967
Fixed delay (Appendix E, Table 3)498.25 ± 0.270.570 ± 0.244
Fixed temporal gate (Appendix E, Table 3)496.09 ± 1.550.163 ± 0.024
Non-adaptive RNN (parameter-matched, H+5H + 5; Appendix E, Table 3)697.80 ± 0.240.410 ± 0.074
New: Sign-free feedback, slow state kept (Appendix J.1, Fig. 18)8 (2 init. × N=3,4N = 3, 4)97.2–98.50.31–0.61
New: Fast negative feedback (θ\theta as fast as hh; Appendix J.1, Fig. 18)4 (N=3,4N = 3, 4)96.9–97.80.55 / 0.89–0.94*

*Oscillates at roughly 3×3\times the task frequency (period ≈T3/3=18–21\approx T_3 / 3 = 18\text{–}21 steps) with degraded long-delay generalization (58.0–72.5%58.0\text{–}72.5\% vs. 84.8–88.6%84.8\text{–}88.6\% for A-RNNs; see Q3 and Appendix J.1).

Furthermore, Appendix H.2–H.4 (Figs. 14–16, Table 5) replicates the phase code, time/content cell populations, and connectivity structure across 1212 independently trained A-RNNs varying in N∈{3,4,5}N \in \{3, 4, 5\}, T3∈{60,65,70}T_3 \in \{60, 65, 70\}, and H∈{450,500,550}H \in \{450, 500, 550\}:

  • 9/129/12 networks develop stable cross-trial OPC (Rϕ>0.9R_\phi > 0.9; overall accuracy 97.90±0.85%97.90 \pm 0.85\%, Rϕ=0.796±0.314R_\phi = 0.796 \pm 0.314).
  • New (Appendix K, K.1–K.2, Fig. 19): Across 4949 additional one-hot A-RNNs (no CNN, non-negative readout, N=4–10N = 4\text{–}10), 49/4949/49 encode serial position by phase (43/4943/49 passing strict concentration and spacing criteria).

Because 0/140/14 ablation runs and 0/80/8 sign-free state-matched controls develop stable OPC (Rϕ>0.9R_\phi > 0.9), these multi-seed results confirm that all three factors are essential for generating stable OPC (and we have updated "necessary" in the Introduction to "required for ... in this working memory task" to avoid any formal-logic ambiguity).

W3/Q2. Does recall causally rely on the phase code? Targeted perturbations

First, the submission already includes a direct causal intervention on the phase readout: shifting the learned temporal gate away from the target phase while holding digit logits fixed drops recall accuracy from 95.7–98.7%95.7\text{–}98.7\% to 16.7–29.1%16.7\text{–}29.1\% across 1212 networks (Appendix H.2, Fig. 14b; see also Section 3.1, Fig. 2e). Pathway-removal (connection-pruning) analyses in Appendix C.3 likewise confirm distinct input/output routing for time and content populations.

AuthorsOpenReview ↗

Response to Reviewer yod7 (Part II/III)

Second, to test directly whether recall relies on internal phase relationships while preserving item representations (Q2), we performed the exact perturbation requested by the reviewer (Appendix C.3). During the delay (at t=120t = 120 or t=155t = 155, one cycle before the cue at step 210210), we advanced selected neuronal populations along their own single-trial trajectories by 1/41/4, 1/21/2, or 3/43/4 of a cycle (+14,+28,+41+14, +28, +41 steps)—shifting phase while preserving amplitude—and then let the network evolve freely (2,0482,048 trials; unperturbed baseline 97.51%97.51\%; results for the half-cycle shift at t=155t = 155 shown below):

Neurons shifted (Appendix C.3)AccuracyOrder swaps among errorsCued-item amplitude ratio (perturbed / intact)
All 500 neurons (global time shift)97.6%12.0%1.03
74 time cells only13.5%96.8%1.02
370 content cells only0.0%95.4%0.97
74 random non-time neurons96.8%15.4%0.84

Selectively disrupting the relative phase between the time-cell reference and content cells abolishes recall, and 95–97%95\text{–}97\% of the induced errors are pure serial-order swaps while the cued item's readout amplitude remains intact (0.97–1.020.97\text{–}1.02). By contrast, applying the exact same shift globally to all neurons—or to 7474 random non-time neurons—leaves recall intact (97.6%97.6\% and 96.8%96.8\%). Full results across +14,+28,+41+14, +28, +41 steps and both intervention times (t=120,155t = 120, 155), as well as pathway-removal experiments, are reported in Appendix C.3.

W4. Role and scope of the circuit model

The circuit model (Section 3.4; Appendix F, F.1–F.4, Figs. 10–11) is a constructive, biologically grounded implementation inspired by the functional organization uncovered in the trained RNN, rather than a model of synaptic learning. Built from established neurobiological motifs—continuous-attractor dynamics, spike-frequency adaptation, and NMDA-like activity-dependent coupling—it demonstrates that the time-ring plus content-assembly architecture can sustain and read out OPC in a mechanistic neural circuit across a broad connected parameter region (Appendix F.4, Fig. 11), yielding concrete experimental predictions (Appendix I; highlighted as a strength by Reviewer medk). At the functional level, the A-RNN itself shows that this organization emerges from task optimization alone; how biological circuits realize and learn these cellular and synaptic couplings remains an open question and is discussed in Section 4.3 (Limitations and Future Directions).

We disagree that the absence of a biological learning rule is a weakness of the circuit model. This objection conflates two distinct explanatory targets: how a circuit performs a computation, and how an organism acquires that circuit. The former has been studied without requiring experimental verification of the latter in models like grid-cell path integration (Burak & Fiete, 2009, PLoS Computational Biology), relational memory (Whittington et al., 2020, Cell), hippocampal phase precession (Chu et al., 2024, eLife), and grid-cell modularity (Khona et al., 2025, Nature), and etc.

Q3. State- and parameter-matched non-adaptive controls

Because each A-RNN unit has two state variables (hh and θ\theta, totaling 1,0001,000 states and 1,029,8751,029,875 parameters at N=4,H=500N = 4, H = 500), our original four-item non-adaptive control (502 hidden units; Appendix D) approximately matched the parameter budget but not recurrent states. We therefore trained controls with the same number of recurrent states and nearly the same number of trainable parameters (1,0001,000 states, 1,030,3751,030,375 parameters at N=4,H=500N = 4, H = 500, plus N=3,H=400N = 3, H = 400; Appendix J.1, Fig. 18a–g) that retain the second state θ\theta and modify only the feedback structure:

Model (N=3N = 3 / N=4N = 4, 3 seeds each at N=4N = 4; Appendix J.1)Feedback typeLearned gain γ⋅m\gamma \cdot m (median)Robust RϕR_\phiPeriodLong-delay accuracy (T2=200–300T_2 = 200\text{–}300)
A-RNN (paired seed)Slow negative1.13 / 1.480.97 / 0.96≈T3\approx T_3 (60 / 55)84.8% / 88.6%
Sign-free feedback (positive init.)Learned, unconstrained sign−0.03 / 0.000.43 / 0.40–0.61None at task scale†67.0% / 68.9–80.1%
Sign-free feedback (zero init.)Learned, unconstrained sign0.00 / 0.000.44 / 0.31–0.39None at task scale†55.8% / 53.8–59.2%
Fast negative feedback (θ\theta as fast as hh)Fast negative (learnable rate)1.07 / 1.280.55 / 0.89–0.94≈T3/3\approx T_3 / 3 (18–21)63.0% / 58.0–72.5%
Non-adaptive RNN (H+5H + 5)None—0.35 / 0.22None at task scale†79.7% / 63.0%

(†Spectral peak falls in the lowest resolvable frequency bin of the analysis window, indicating no oscillation at the task timescale.)

AuthorsOpenReview ↗

Response to Reviewer yod7 (Part III/III)

  • Sign-free feedback: When gradient descent is free to learn the feedback sign, it never discovers negative feedback—the learned gain collapses to near zero (median γ⋅m≈0\text{median }\gamma \cdot m \approx 0). Although these networks learn in-distribution recall quickly and accurately (97.2–98.5%97.2\text{–}98.5\%; zero-init reaches 80%80\% validation accuracy at epochs 8–98\text{–}9 vs. epoch 1010 for the A-RNN; Fig. 18g), they develop no task-timescale oscillation or OPC (Rϕ=0.31–0.61R_\phi = 0.31\text{–}0.61; Fig. 18b,c,e), keep their readout gate pinned to the last retrieval step like vanilla RNNs, and generalize poorly to unseen long delays (53.8–80.1%53.8\text{–}80.1\% vs. 84.8–88.6%84.8\text{–}88.6\%; Fig. 18f).
  • Fast negative feedback: Making θ\theta as fast as hh produces a rhythm roughly three times faster than the retrieval window (period≈T3/3=18–21\text{period} \approx T_3/3 = 18\text{–}21 steps; Fig. 18d), so each phase recurs multiple times within T3T_3, slowing learning (reaching 80%80\% at epochs 15–3215\text{–}32; Fig. 18g) and degrading long-delay generalization (58.0–72.5%58.0\text{–}72.5\%; Fig. 18f).
  • Conclusion: Adding an extra slow state variable by itself does not explain OPC or its long-delay generalization advantage; slow negative feedback (spike-frequency adaptation) is the essential mechanism, fully confirming our original conclusion (Appendix J.1).

Q4. Repeated digits and sequence length

  • Sequence length: OPC emerges consistently across N∈{3,4,5}N \in \{3, 4, 5\} in our 1212 multi-configuration CNN-A-RNNs (Rϕ>0.9R_\phi > 0.9 in 3/33/3 three-item, 5/65/6 four-item, and 1/31/3 five-item networks; Appendix H.2, Table 5, Fig. 14) and across N=4–10N = 4\text{–}10 in all 49/4949/49 one-hot A-RNNs (Appendix K, K.1–K.2, Fig. 19).
  • Repeated digits (Appendix J.2, Fig. 18h–m): In our standard task, digits represent distinct abstract items (following standard serial-recall paradigms without replacement). To test how the phase code handles item repeats, we trained new N=4N = 4 networks with sampling with replacement (49.6%49.6\% of trials contain repeats; 2 seeds):
    1. High recall accuracy & human-like Ranschburg cost: Repeat-trained networks achieve 97.1–97.2%97.1\text{–}97.2\% on standard no-repeat trials (main network: 97.8%97.8\%) and 90.5–95.3%90.5\text{–}95.3\% on repeat trials, with slightly lower accuracy when the cued item itself is the repeated digit (87.2%/94.2%87.2\% / 94.2\%) than when a unique digit in the sequence is cued (94.5%/96.6%94.5\% / 96.6\%), mirroring the classic Ranschburg effect in human serial recall.
    2. Zero-shot prediction on adjacent repeats vs. learned gate alignment on non-adjacent repeats: Even the original main network (never trained on repeats) generalizes zero-shot when a repeated digit occupies adjacent positions (91.3%91.3\%), though its accuracy drops on non-adjacent repeats (63.9%63.9\%; Fig. 18h,i,l). Training with replacement equips the temporal gating head with much stronger alignment to the repeated digit's retrieval peak across different ordinal cues (gate-to-peak alignment within 3 circular steps more than doubles from 16.4%16.4\% to 33.2–36.1%33.2\text{–}36.1\%; Fig. 18m), thereby successfully resolving non-adjacent repeats as well (94.8–95.1%94.8\text{–}95.1\%; Fig. 18j–l).
    3. Distinct-item maintenance during the delay (single peak per cycle): Crucially, in both the untrained and repeat-trained networks, the delay-period OPC maintains mutually distinct digit identities—a repeated digit exhibits only a single prominent readout peak per oscillation cycle (86.9%86.9\% in the distinct-only model and 93.8–98.1%93.8\text{–}98.1\% in the repeat-trained models; Fig. 18m) rather than peaking twice within a cycle, while position disambiguation at retrieval is handled by the cue-dependent gating head.

Summary on informativeness

In summary, our existing analyses and new experiments directly address each of your questions: we have clarified the quantitative cell-classification criteria (W1/Q1), reported full multi-seed statistics across all ablations and 6161 trained A-RNNs (W2/Q5), demonstrated via targeted phase-shift perturbations that recall causally relies on the relative phase between time and content cells (W3/Q2), clarified the constructive role of the biological circuit model (W4), and verified that OPC requires slow negative feedback rather than extra state variables (Q3) and extends robustly across sequence lengths (N=3–10N = 3\text{–}10) and repeated items (Q4). We hope these clarifications and new experiments fully resolve your concerns and would greatly appreciate a reassessment of our work.

AuthorsOpenReview ↗

Reminder to reply

Dear Reviewer, We kindly ask you to consider our responses and would be happy to address any further questions. As the author–reviewer discussion period ends on November 18 (AoE), we would greatly appreciate it if you could reply at your earliest convenience.

Thank you for your time and constructive feedback.

Reviewer_aR9COpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors train an RNN with adaptation on a sequential working memory task. Four items are presented one after the other. This is followed by a variable delay period. A retrieval period then provides a one-hot encoding of which serial position to read out, and the network outputs the item in that position. The authors show that the network develops an oscillatory phase coding. The neurons oscillate with a frequency that corresponds to the duration of the retrieval period, and the one-hot input is translated into a gating that selects the correct phase. The authors further show that variable delays, adaptation and the flexible readout architecture are all essential. They also characterize time- and content- single units, and propose a simplified network inspired by their findings.

Critical strengths

This is a clear demonstration of how oscillatory phase coding can arise despite the absence of oscillations in the task design (the input can be thought of as oscillatory, but the period of 10 is not the one the network develops). The ablation controls nicely show that the various components are required. The single cell findings were surprising (to me), and seemed quite strong.

Critical weaknesses

While the authors demonstrate that the various components are needed to obtain OPC, they don’t explain why this is the case. There are studies that show that oscillations can arise in RNNs (Orhan and Ma, Turner and Barak, even though there fixed delays encouraged oscillations). It would strengthen the papers if the authors go beyond descriptive accounts to mechanistic accounts of why learning converges to this solution. Is the readout (equation 4) causal? If I understand correctly, the fixed gating control (Fig 3Aii) used a readout that is more causal. It isn’t fully clear if this control can still do the task. In principle, one can imagine the same oscillatory memory using phases – but the ordinal cue causing a phase shift during stage 3. Does this indeed happen? The reduced model is inspired by the time and content cells, but it is not a reduction of the full model.

Orhan, A. Emin, and Wei Ji Ma. “A Diverse Range of Factors Affect the Nature of Neural Representations Underlying Short-Term Memory.” Nature Neuroscience 22, no. 2 (2019): 2. Turner, Elia, Kabir V. Dabholkar, and Omri Barak. “Charting and Navigating the Space of Solutions for Recurrent Neural Networks.” Advances in Neural Information Processing Systems 34 (2021): 25320–33.

Questions for authors

Line 265: something is strange in the first sentence. Figure 4: How many cells are neither time nor content?

2 comments
AuthorsOpenReview ↗

Response to Reviewer aR9C (Part I/II)

We thank the reviewer for the positive assessment and insightful comments.

W1-a. Why does learning converge to OPC? (Conditions for convergence and training dynamics)

  1. Limitations of BP training dynamics and future directions: We fully agree that how backpropagation shapes recurrent weights over training epochs remains largely a black box. While our ablations identify under what conditions optimization converges to OPC, characterizing how the phase code forms along the learning trajectory is an important direction for future work—specifically, applying dimensionality reduction (e.g., PCA/dPCA/UMAP) and topological data analysis (TDA, such as persistent homology) across training checkpoints to track how a ring-like limit cycle and phase separation emerge during learning.
  2. Why task learning prefers the OPC solution:
    • OPC is a computationally superior solution for flexible sequence working memory: Compared with memorizing separate fixed-point or trajectory attractors for every possible sequence combination, OPC (i) disentangles working-memory structure ("where/when") from content ("what") via phase and amplitude, (ii) provides compositional representations that dramatically boost memory capacity, and (iii) generalizes naturally across variable delay durations because relative phases on a shared limit cycle remain invariant over time (Appendix G, Fig. 12).
    • SFA and flexible readout act as inductive biases guiding BP toward OPC: Slow adaptation destabilizes static fixed-point maintenance in favor of slow oscillations, variable delays penalize fixed-time ramping solutions, and the learnable temporal gate enables phase-selective retrieval—jointly steering gradient-based optimization toward this high-capacity, delay-invariant OPC solution. Notably, in our early experiments, we observed that when slow negative feedback is present (γ=1\gamma = 1), task optimization also tends to actively increase the learnable adaptation gain mim_i during training (even when initialized weakly at mi=0.05m_i = 0.05, driving units across the threshold into the damped oscillatory regime.

W1-b. Is the readout (Eq. 4) causal?

Pairing between coding and readout (causal role in OPC emergence): This is a very sharp insight—neural coding schemes and readout mechanisms are inherently paired. Because an oscillatory phase code carries different items at distinct phases rather than at a fixed time step, the network must be able to autonomously select which phase to read out; in this sense, the flexible temporal gating readout in Eq. 4 is indeed causally required for the emergence of OPC (Section 3.2, Fig. 3a(ii),c). Conversely, a hypothetical strategy of combining a cue-driven phase shift with a fixed readout fails to decouple the cue (structural query) from the maintained memory representation (OPC), making it less efficient than a flexible readout (see W1a-2 above) and preventing it from winning out during optimization.

W1-c. Does the cue phase-shift the oscillation in the fixed-gate network?

First, the fixed-gate network can perform the task (96.09±1.55%96.09 \pm 1.55\% accuracy across 44 seeds; Appendix E, Table 3), but it does not adopt an oscillatory phase code (Rϕ=0.16±0.02R_\phi = 0.16 \pm 0.02; Section 3.2, Fig. 3a(ii),c). Second, regarding whether the cue induces a phase shift: no—replaying identical sequences under different ordinal cues shows that the fixed-gate network does not phase-shift an ongoing oscillation; instead, it relies on a cue-driven transient readout strategy, where the cue triggers a transient ramp that pushes the queried item to peak specifically at the fixed readout window at the end of T3T_3 (see Appendix E.1 for details).

W1-d. The circuit model is not a mathematical reduction of the full RNN

We completely agree—and this was our exact intention from the outset:

  1. A newly constructed inspiration model, not an RNN reduction: As stated in our Introduction (lines 082–084: "Inspired by these findings, we proposed a neural circuit model that reproduces OPC through interactions between content units and an adaptive time ring, providing a simple account of how OPC could operate in biological circuits") and Section 3.4 title ("A Neural Circuit Model of OPC Inspired by the Trained A-RNN"), the circuit model is a new mechanistic model inspired by the functional organization uncovered in the trained A-RNN, rather than a mathematical reduction of the RNN.
AuthorsOpenReview ↗

Response to Reviewer aR9C (Part II/II)

  1. Why build a new theoretical circuit model when we already have the trained A-RNN? In fact, the A-RNN training and analyses in the first part of our paper serve as the foundation to motivate this second part. Because artificial neural networks can be parameterized in countless ways, reverse-engineering the idiosyncratic micro-wiring of a specific artificial RNN is not an end in itself; what matters for neuroscience is explaining how biological neural circuits operate. Our motivation for constructing a new theoretical circuit model is therefore to explain a plausible biological circuit mechanism for implementing OPC in working memory (via an adaptive continuous-attractor ring, content assemblies, and NMDA-like gated coupling) and to generate falsifiable biological predictions.
  2. Manuscript revisions for maximum clarity: We appreciate your pointing this out—if this motivation was not immediately transparent, the responsibility lies with our phrasing (for instance, referring to "Our simplified neural circuit model" in Section 4.3, line 501, which could be misread as a simplified reduction of the RNN). We have therefore revised Section 3.4 (lines 449–450), Section 4.1 (lines 460–462), and Section 4.3 (line 501) to state explicitly that the model is a "mechanistic neural circuit model of biological OPC inspired by—rather than reduced from—the trained A-RNN, constructed to provide a transparent account of how biological circuits can implement and sustain OPC."

Q1. Wording on Line 265

Thank you for catching this. The original sentence opened a paragraph with a symbol before defining it in prose. We have revised it in Section 3.2 to read: "To compare these networks quantitatively, we define a phase-consistency index RϕR_\phi, which measures how consistently the phase differences between items at different serial positions are preserved across trials; its robust version takes the 20th percentile over temporal windows (Appendix D). Rϕ=1R_\phi = 1 indicates perfectly stable phase relationships."

Q2. Figure 4: Number of unclassified cells

Out of the 500500 hidden units, 7474 are classified as time cells, 370370 as content cells, and 5656 are unclassified (Appendix C). Figure 4c now labels the content and unclassified cell counts, and the Figure 4 caption reports all three counts.


We hope these clarifications and the new cue-replay and multi-seed analyses fully address your questions, and we greatly appreciate your recognition of our work's core contributions—demonstrating that OPC emerges without oscillatory inputs or supervision, establishing the minimal task and biophysical conditions required through systematic ablations, uncovering the single-cell division into time and content cells, and proposing a biological neural circuit model that yields testable predictions.

Reviewer_medkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper shows that Oscillatory Phase Coding (OPC), a phenomenon observed in the brain especially during navigation, naturally emerges in artificial adaptive RNNs. They train their RNN on a sequence memory task with variable delays using the MNIST dataset and a CNN for image encoding and an MLP for cue encoding. They identify three key factors that are necessary for the emergence of OPC: spike frequency adaptation, having a flexible readout time, and having variable delays. Information is encoded using the amplitude, which corresponds to the item identity, and phase, which corresponds to the serial position. They identify two different cell types: time cells forming a traveling wave that provides a temporal reference frame, and content cells that encode the item identity. Based on this, they build a neural circuit model with time cells forming a ring attractor network with adaptation and one content unit per digit-selective content cell. They predict that changing the item's serial order in a sequence will shift the phase of corresponding content cells while preserving the phases of time cells.

Critical strengths

1. The primary strength of this work is demonstrating that OPC is reproducible in adaptive RNNs. The authors show that when an A-RNN is trained on a flexible working memory task with variable delays, it spontaneously develops oscillatory dynamics where memorized items recur at distinct phases. This is a compelling computational result that links neural adaptation directly to the emergence of phase-based coding schemes.

2. Furthermore, the study identifies a clear mechanistic separation between time cells and content cells. Content cells appear to encode item identity, with phases tracking serial position, while time cells form a traveling wave that provides a shared temporal reference. This decomposition provides a testable hypothesis about how biological systems might implement phase coding. The authors also generate testable predictions through their proposed neural circuit model, suggesting that changing an item's serial position should shift the phase of corresponding content cells while preserving the temporal reference provided by time cells. These predictions could guide future experimental work.

Critical weaknesses
  1. Oscillatory phase coding plays a particularly prominent role in spatial navigation, where hippocampal place cells exhibit theta phase precession. As animals navigate through space, place cells sequentially activate at different theta phases. The paper trains an RNN on a working memory task rather than a navigation task, which means the study provides no insight into how OPC influences phase coding during spatial navigation or what computational advantage it confers in that context.
  2. In addition to the point above, there are decade-old studies on the neural function of OPC. Given the authors use an RNN-based approach, at least a better conceptual connection with such literature needs to be established (e.g., see https://www.sciencedirect.com/science/article/pii/S0896627309000750?via%3Dihub and works by Kayser, Brunel and others). Along the same line, traveling waves also gained currency recently (https://www.sciencedirect.com/science/article/pii/S0896627326004873?via%3Dihub) that are well-connected to RNNs (see https://proceedings.iclr.cc/paper_files/paper/2024/hash/374fc6371864f800f213bf8a248f1117-Abstract-Conference.html and https://proceedings.mlr.press/v202/keller23a.html).
  3. The study provides no neural verification of its theories. Despite substantial neurophysiological evidence for oscillatory coding in biological systems, particularly in the hippocampus, the entire analysis relies on RNN simulations. There is no comparison to existing cell types or real neural data (at least at a conceptual level). The paper identifies "time cells" and "content cells" based on activity patterns and connectivity, but these labels are post-hoc classifications. The authors do not show how their emergent cell types correspond to known biological populations.
  4. The claim that OPC "emerges spontaneously" is misleading. The authors explicitly trained the network on a task requiring sequence working memory with variable delays, and they incorporated adaptation into the architecture. The emergence of OPC is therefore architecturally and task-dependently driven, not truly spontaneous. The fact that SFA can lead to oscillations (as shown in Chu et al., 2024; Ji et al., 2025) makes the emergence of OPC in the A-RNN less surprising. The combination of SFA with a variable-delay memory task essentially engineers the desired dynamics rather than revealing a purely spontaneous phenomenon.
  5. The A-RNN architecture is a hybrid design that lacks biological plausibility in several ways:
    The network includes a CNN input encoder for processing handwritten digits. This makes the architecture unnecessary, complicated, and more distant from biology. The paper provides no discussion of neural sparsity, Dale's Law, or other biological constraints.
  6. The choice to use MNIST handwritten digits is questionable from a biological perspective. MNIST is a computer vision benchmark, not a biologically-relevant stimulus. Handwriting recognition requires an elaborate CNN encoder in the simulation, which introduces additional artificiality. Biologically, animals encounter visual scenes, objects, or spatial trajectories, not handwritten numbers. If the goal is to draw parallels to hippocampal navigation, the authors should simulate a task more ecologically valid for navigation, such as traversing a virtual environment or following a learned path. Using a simpler representation (e.g., geometric shapes, path coordinates, or binary sequences) would make the architecture cleaner and reduce the need for complex preprocessing. The CNN appears to act as a "fix" to make the problem solvable rather than reflecting how biological systems naturally encode sensory inputs.
  7. “Adaptation was motivated by previous studies showing that spike-frequency adaptation is necessary to generate oscillatory dynamics associated with the hippocampal theta phase precession” (Chu et al., 2024; Ji et al., 2025). These papers show spike-frequency adaptation as one possible mechanism to create OSC. They do not show that it is necessary.
Questions for authors
  1. What is the optimal oscillation frequency? A deeper analysis of the frequency would be interesting.
  2. Would the oscillation frequency depend on RNN parameters, and why?
  3. Why was MNIST chosen as a data set and not a more biologically relevant data set?
  4. What happens in biologically more realistic architectures?
  5. Oscillation in the brain, which inspired the framework discussed in this paper, is ubiquitous (across tasks, animals, etc) (https://www.sciencedirect.com/science/article/pii/S0896627313009045?via%3Dihub). The conclusion in the paper is fairly general but based on limited cases (also see point 4 above). Given that, how do the authors assess the “emergence” of oscillations in the RNN to be functional?
5 comments
AuthorsOpenReview ↗

Response to Reviewer medk (Part I/IV)

We thank the reviewer for the constructive comments and for recognizing our findings as a "compelling computational result" that provides a "testable hypothesis" for future experimental work.

W1. Navigation vs. sequence working memory

  1. Scientific importance and our theoretical & mechanistic contributions to OPC in working memory: Investigating the relationship between working memory and oscillatory phase coding (OPC) is in itself a central, long-standing question in neuroscience. While extensive experimental and theoretical studies have implicated OPC in sequence working memory—proposing that sequential items are maintained and read out at distinct phases of a shared oscillation (e.g., Lisman & Idiart, 1995; Lisman & Jensen, 2013; Siegel et al., 2009; Bahramisharif et al., 2018; Han et al., 2026)—how such a code arises from learning and operates in neural circuits has remained largely speculative. Our work provides both a normative theoretical proof from an optimization perspective (identifying the minimal task and biophysical conditions under which OPC emerges through training) and a concrete mechanistic circuit model explaining how biological circuits can implement and sustain OPC via interacting time and content populations.
  2. Connection to spatial navigation and future extensions: Although this paper is not aimed at explaining phase precession in spatial navigation, it is not entirely disconnected from navigation—(indeed, spatial navigation inherently requires working memory, and prominent studies argue that hippocampal place-cell theta sweeps and phase coding act as a working-memory-like process that sequentially samples and holds surrounding environments and candidate trajectories, e.g., Johnson & Redish, 2007; Wikenheiser & Redish, 2015; Buzsáki & Moser, 2013; Kay et al., 2020; moreover, both share spike-frequency adaptation as a core biophysical mechanism, Chu et al., 2024; Ji et al., 2025; Section 4.2). Extending our framework to navigational phase coding is a natural direction for future work.
  3. Spatial navigation is out of scope and not a valid weakness of this study: We state firmly that our paper explicitly focuses on sequence working memory, not spatial navigation. Navigation is a separate research topic that lies outside the scope of the present study; faulting a working-memory paper for not solving an unrelated topic outside its defined scope is unwarranted and should not be considered a weakness of our work.

W2. Connection to the phase-of-firing and traveling-wave literature

Thank you for pointing us to these important references. We have integrated them into the Related Works section of the Discussion (Section 4.2, Multiplexed phase codes and traveling waves, p. 10, lines 489–497) along two axes:

  1. Multiplexed phase-of-firing codes (Mazzoni et al., 2008; Kayser et al., 2009; Panzeri et al., 2010): Our trained A-RNNs realize precisely the multiplexed coding principle established in these studies, where firing-rate amplitude encodes item identity ("what") while spike/activity phase relative to a slow oscillation encodes serial order ("when"), boosting information capacity and robustness.
  2. Cortical traveling waves (Muller et al., 2026, Neuron) and wave-based RNNs (Keller & Welling, 2023; Keller et al., 2024): A critical distinction is that wave-RNNs hard-code traveling waves into the architecture via predefined spatially local connectivity. By contrast, our A-RNN imposes no prior recurrent connectivity structure: both the traveling wave of time cells (Fig. 4d) and its approximately cosine-shaped recurrent kernel (Fig. 4e) emerge through task training alone.

W3. Biological correspondence and whether time/content cells are arbitrary post-hoc labels

  1. The model reproduces an observed signature of working memory: Our work is not a purely abstract RNN analysis, but a computational modeling study explicitly grounded in the widely observed neurobiological phenomenon of oscillatory phase coding in working memory (where memorized items recur at distinct phases of a shared oscillation; see Section 4.2, Theory of OPC, p. 9, lines 468–470). Reproducing this empirical signature from task and biophysical constraints already provides a direct bridge to biological recordings.
AuthorsOpenReview ↗

Response to Reviewer medk (Part II/IV)

  1. Biological counterparts of time and content cells: Both identified populations have clear biological correspondences, distinguishing what is experimentally established from our model's new prediction:
    • Time cells (model prediction with hippocampal parallels): Periodic time cells that provide an item-independent oscillatory phase reference in sequence working memory are a core prediction of our model (as noted in Strength 2). They closely parallel hippocampal and entorhinal time cells that tile delay intervals sequentially (MacDonald et al., 2011; Eichenbaum, 2014), and we provide a concrete experimental protocol to test them (Appendix I): changing an item's serial position should shift the preferred phase of the corresponding content-selective neurons while leaving the phase progression of time cells invariant.
    • Content cells (biologically observed item-selective phase-locked neurons): Unlike periodic time cells, "content cells" directly correspond to stimulus- and item-selective neurons documented in biological working-memory recordings—for example in (Kamiński et al., 2020, Neuron), as well as (Siegel et al., 2009, PNAS); see also (Bahramisharif et al., 2018; Rutishauser et al., 2010). Because experimental studies typically refer to these neurons by their specific task stimuli (e.g., "image-selective," "object-selective," or "letter-selective" neurons) without a single unified name, we adopt the concise functional term "content cells" to refer to item-selective neurons and contrast them cleanly with item-independent "time cells." Meanwhile, whether shifting an item's serial position shifts the preferred firing phase of the same item-selective neuron (while leaving time-cell phases invariant) is the specific mechanistic prediction tested by our proposed experiment in Appendix I.
  2. Empirical validation that time/content cells reflect a genuine functional division rather than arbitrary post-hoc labels:
    • Robustness across thresholds and seeds: Cell populations and connectivity kernels remain stable across a 63-point threshold grid (Appendix C.2, Fig. 7), split-half Jaccard indices across trials are 0.94–0.990.94\text{–}0.99, and the same bimodal organization recurs across 99 independently trained configurations (Appendix H.3–H.4, Figs. 14e,f, 16).
    • Robustness to input modality and sign constraints (new, Appendix K.1): In networks trained without a CNN using 25-category one-hot inputs and non-negative readout weights, the exact same classification criterion identifies 39–4339\text{–}43 time cells and 422–445422\text{–}445 content cells tiling all 2525 items.
    • Causal functional dissociation (new, Appendix C.3): Advancing only the 7474 time cells or only the 370370 content cells along their delay trajectories by half a cycle (changing phase while preserving amplitude) collapses accuracy from 97.5%97.5\% to 13.5%13.5\% and 0.0%0.0\% via 95–97%95\text{–}97\% pure serial-order swaps while preserving the cued item's readout amplitude (0.97–1.020.97\text{–}1.02), whereas shifting all neurons together (97.6%97.6\%) or shifting 7474 random non-time neurons (96.8%96.8\%) has no effect. Pathway-removal analyses in Appendix C.3 likewise confirm distinct input/output routing for the two populations.
  3. Standards across top ML and neuroscience venues: Both in top machine learning conferences (ICLR, NeurIPS) and in top-tier neuroscience and general-science journals, theoretical and computational modeling studies are routinely published without validating their predictions against newly collected neural recordings—for instance, in short-term memory (Orhan & Ma, 2019, Nature Neuroscience) and spatial navigation (Banino et al., 2018, Nature; Cueva & Wei, 2018, ICLR; Chu et al., 2025, NeurIPS). Our work grounds its model in established OPC neural phenomenon, and provides falsifiable experimental predictions and protocols for future neurophysiological testing (Appendix I). Accordingly, the absence of new biological recording verification is, by itself, not a valid ground for rejection.

W4. Why SFA alone does not yield OPC, and the meaning of "emergence" in the AI-for-Neuroscience paradigm

We address this concern on two levels: first, empirically and theoretically, incorporating SFA neither automatically produces OPC nor even guarantees oscillations; second, methodologically, identifying the task and biophysical conditions under which a neural code emerges through optimization is a core scientific paradigm in AI for Neuroscience rather than "engineering" the solution.

AuthorsOpenReview ↗

Response to Reviewer medk (Part III/IV)

  1. Adding SFA neither automatically produces OPC nor even guarantees oscillations:
    • SFA does not automatically yield OPC: In our ablations, A-RNNs equipped with the exact same SFA mechanism and initialized in the oscillatory regime still exhibit oscillatory dynamics under a fixed delay or a fixed temporal gate, yet they fail to organize those oscillations into a stable phase code (Rϕ=0.57±0.24R_\phi = 0.57 \pm 0.24 and 0.16±0.020.16 \pm 0.02 across 4 seeds each, vs. 0.970.97 in the main network; Section 3.2, Fig. 3; Appendix E, Tables 3–4).
    • Adding an adaptation variable does not even guarantee oscillations—it requires a specific parameter regime: As proved by our analytical Jacobian derivation for a single adaptive unit (Appendix H.2, Eqs. 38–39), introducing an adaptation state only produces intrinsic oscillations when the parameters lie within a specific regime (for a damped oscillatory mode, Eqs. 38–39). Our state-matched controls (Appendix J.1, Fig. 18a–g) confirm this empirically: even when the adaptation parameters are fully learnable, if the network is initialized in a non-oscillatory region (nearly zero or positive feedback initialization), gradient descent does not enter the oscillatory regime (median γ⋅m≈0\text{median }\gamma \cdot m \approx 0, Rϕ=0.31–0.61R_\phi = 0.31\text{–}0.61) and instead converges to a non-oscillatory, vanilla-RNN-like solution. Thus, merely adding an adaptation variable guarantees neither oscillations nor OPC.
  2. "Emergence" in the normative AI-for-Neuroscience paradigm vs. hand-engineering:
    • Our use of "emerges spontaneously" (a phrasing the reviewer also used in Strength 1) means that neither oscillatory activity, phase-ordered coding, nor the separation into time and content cells is prescribed in the loss function, inputs, or recurrent connectivity (Appendix A.2, Eq. 12)—the network is trained end-to-end via backpropagation solely to report the cued item.
    • More fundamentally, training RNNs under specific task settings and biological constraints to determine when a brain-like representation arises is the standard normative methodology of AI for Neuroscience (see Richards et al., 2019; Yang & Wang, 2020; Doerig et al., 2023; Golan et al., 2026), not ad-hoc engineering. Uncovering the minimal ecological and biophysical constrains under which parameter learning reliably and robustly converges to OPC is itself a contribution to neuroscience, as it can provide insights to why and under what conditions neural circuits may adopt phase coding. If the reviewer still feels that "spontaneously" could be misread despite this standard usage, we would be happy to qualify it as "emerges without rhythmic input or phase supervision" in the text (as already phrased in Section 4.1 and Section 4.2).

W5/W6/Q3/Q4. Role of the CNN encoder, choice of MNIST, and biological constraints

In our setup, the CNN serves purely as a feed-forward sensory front end—itself loosely inspired by early visual cortex—to map pixel inputs into feature vectors, while our scientific focus is on the recurrent memory dynamics of the A-RNN (Section 2.2; Appendix A.1). Digits represent distinct abstract items to be held in working memory, directly analogous to stimulus sets in human and primate serial-recall paradigms. MNIST is widely used across NeuroAI studies of biologically plausible learning, spike-frequency adaptation, and working memory (e.g., Lillicrap et al., 2016; Guerguiev et al., 2017; Masse et al., 2018; Bellec et al., 2018; Salaj et al., 2021; Payeur et al., 2021; Ghazizadeh & Ching, 2021; Geadah et al., 2024) precisely because its stimuli provide natural within-class variation without confounding color or background texture, making the memorized content unambiguous to decode and interpret.

To demonstrate directly that our findings do not depend on the CNN or on unconstrained readout signs, we trained 49 new A-RNNs without the CNN, using 25-category one-hot inputs and non-negative readout weights (interpretable as non-negative firing rates; Appendix K, K.1–K.2, Fig. 19). Across all 49 networks (N=4–10N = 4\text{–}10), oscillation phase selectively encodes serial position (median circular variance explained 0.968–0.9920.968\text{–}0.992 by position vs. 0.035–0.0520.035\text{–}0.052 by item identity; 43/4943/49 meeting our strict OPC spacing criterion). All 21 networks trained at N=4N = 4 satisfy the strict OPC criterion with 99.99–100%99.99\text{–}100\% test accuracy. In the three networks examined at the single-cell level, we identified 39–43 time cells and 422–445 content cells. All core conclusions of the paper therefore hold without the CNN and under non-negative readout constraints.

Comments on the paper

1 comment
AuthorsOpenReview ↗

A brief response about causality

Thank you for the clarification! Our wording in the main text created this ambiguity, and we also misunderstood your question about temporal causality in our previous reply. The actual output of the gating head at each time step is the unnormalized variable gtg_t defined in Appendix A.1, Eq. (10); αt\alpha_t is obtained by normalizing these outputs over the entire retrieval window (Eq. (11)). We will revise the wording in Section 2.2 promptly to make this distinction explicit and avoid further misunderstanding as soon as possible.

Yes, the readout is strictly causal in time and well-defined at every time step tt:

  1. Instantaneous readout and gating depend only on the current state (no future dependence): At every time step tt, both the digit neurons readout pt=Woutht+bout\mathbf p_t = W_{\mathrm{out}}\mathbf h_t + \mathbf b_{\mathrm{out}} (Appendix A.1, Eq. (9)) and the unnormalized gating activity gt=wg⊤ht+bgg_t = \mathbf w_g^\top \mathbf h_t + b_g (Appendix A.1, Eq. (10)) are well-defined functions solely of the current hidden activity ht\mathbf h_t (and recurrent state updates never access future states or αt′>t\alpha_{t'>t}; Section 2.2, Eqs. (1)–(3)).
  2. At the end of the T3T_3 decision window: The sum over T3T_3 in Eq. (4) (Section 2.2) simply reflects that the network makes its final trial-level decision at the end of the retrieval period T3T_3 by integrating all ptp_t collected over T3T_3. Moreover, because αt=egt/∑s∈T3egs\alpha_t = e^{g_t}/\sum_{s\in\mathcal T_3}e^{g_s} (Appendix A.1, Eq. (11)) and y^=∑t∈T3αtpt=∑t∈T3egtpt∑t∈T3egt\hat{\mathbf y} = \sum_{t\in\mathcal T_3}\alpha_t\mathbf p_t = \frac{\sum_{t\in\mathcal T_3}e^{g_t}\mathbf p_t}{\sum_{t\in\mathcal T_3}e^{g_t}}, a downstream decision only needs to maintain causal running accumulators At=∑s∈T3, s≤tegsps\mathbf A_t = \sum_{s\in\mathcal T_3,\,s\le t}e^{g_s}\mathbf p_s and Zt=∑s∈T3, s≤tegsZ_t = \sum_{s\in\mathcal T_3,\,s\le t}e^{g_s} up to the current step tt, so the running readout At/Zt\mathbf A_t/Z_t is also causally well-defined at any point in time and yields y^\hat{\mathbf y} once T3T_3 concludes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.