acceptodds
Under review as a conference paper at ICLR 2027

Binding Phoneme Identity to Position in Neural Models and the Human Brain

Abstract

A fundamental challenge in cognitive science and neural computation is the binding problem: how neural systems associate separately represented contents with their structural roles. Compositional generalization can be achieved by factorizing representations into roles, the structural slots of a representation (e.g., serial position or syntactic function), and fillers, the contents that occupy them (e.g., a phoneme or a word). Because roles and fillers can be recombined freely, a system that represents them separately can represent novel combinations. How neural systems implement role–filler binding, however, remains largely unknown. One of its simplest linguistic cases is phoneme sequencing, where phoneme identity (the filler) must be bound to position (the role) so that /gum/ and /mug/ are represented differently. While previous work identified the neural encoding of single phonemes in human auditory cortex, it remains unclear how these encodings are combined into sequences. A recent proposal from natural language processing, the “onion hypothesis” (Csordás et al., 2024), offers a concrete account: the direction of the neural population state encodes token identity, while its magnitude encodes token position. However, the onion hypothesis was formulated for random token sequences; here we extend it to natural language, where successive tokens are strongly constrained, and test it in both neural models and the human brain. We first evaluated the onion hypothesis in an encoder–decoder trained to repeat English words, combining representational geometry analyses with causal interventions. Magnitude alone turns out to be incomplete: position is encoded by magnitude together with a systematic rotation of the state direction, a scheme we formalize as the “spiral hypothesis.” We then analyzed intracranial recordings from the auditory cortex of 20 participants listening to short spoken sentences. Across superior temporal gyrus electrodes, the change in population response between consecutive phonemes of a word decreases in magnitude as position advances, while the direction of that change is positively aligned within consonants and within vowels and close to orthogonal across the two classes. This supports the magnitude coding shared by both the onion and spiral accounts. Together, these results provide a geometric account of how neural systems bind phoneme identity to position, and show how hypotheses derived from neural models can yield testable predictions about human neural computation.

Primary area:
applications to neuroscience & cognitive science
Keywords:
The Binding problem, Auditory Cortex Analysis, Word Repetition, Phonology, Causal Mediation Analysis, Neural Representations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ABPZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the representations of phonemes in RNNs and in the human brain, in particular regarding their position and identity. The authors test the Onion hypothesis, which posits that, in RNNs, the direction of the state encodes token identity and its magnitude encodes token position. They test this hypothesis, alongside other alternatives: their proposed Spiral Hypothesis, which suggests that position is encoded jointly by a decaying magnitude and a systematic state rotation, and subspace baselines. They first test phoneme representations in an RNN trained to repeat English words sampled according to their natural frequency. They show that the norm of the encoding decreases with position, in accordance with the Onion hypothesis. They show that, at a class-level (consonants and vowels), phoneme identity is encoded in direction. However, unlike Onion predictions, they show that identity direction varies with position. Through causal interventions, they show that the Spiral hypothesis achieves higher predictions, in particular for late positions. Last, using intracranial recordings of the STG, they show that phoneme position is encoded in response magnitude, and that the phoneme directions are more aligned within than across class.

Critical strengths
  1. The paper develops a concrete and testable hypothesis of phoneme-position binding, motivated by previous literature where they identified a limitation and thus refined it.
  2. The authors used different methods, causal interventions, comparison with baselines, and investigated RNNs and human brains. Their findings strongly support the Spiral hypothesis in RNNs.
Critical weaknesses
  1. While confounds such as surprisal are taken into account, other low-level features of the phonemes (e.g., duration) are not tested. The comparisons are also restricted to a coarse-level (vowels vs consonants) and not at the individual phoneme level.
  2. The intracranial findings support the magnitude results common to the Onion and the Spiral hypotheses but cannot distinguish them based on identity direction. This distinction is thus not tested in the brain.
Questions for authors

How do you interpret the slight upward bump in Figure 4B? Are particularly long words subject to exceptional phonotactics?

1 comment

We thank the reviewer for the positive assessment of our hypotheses and methods.

Response to Critical Weakness 1:

Low-level confounds. We agree that this is an important control; Reviewer 1 raised a related concern. If position is encoded in response magnitude, its effect should survive when low-level features are controlled for. We therefore regressed ‖Δ‖ on position together with surprisal, lexical stress, phoneme duration, sound level and the spectral change between consecutive phonemes, with standard errors clustered by word (full table in our response to Reviewer 1). Position remains significant (p = 1.7×10⁻⁶). Duration and sound level have the largest effects: later phonemes tend to be longer and quieter, which accounts for part, but not all, of the decrease with position. In the revised manuscript, Section 3.3 now reports this regression, with the full analysis in a new Appendix A.4.

Individual phonemes. We thank the reviewer for this suggestion, which led to a new result. We added finer-grained comparisons among consonants, which have enough tokens per phoneme. Pairs of the same consonant are more aligned than pairs of different consonants (AUC = 0.52, p < 0.001). Among different consonants, pairs that share their manner of articulation are more aligned than pairs that do not (AUC = 0.54, p = 0.01; permutation test shuffling manner labels across phonemes), whereas place of articulation shows no effect (AUC = 0.49, p = 0.42). The direction of the neural response therefore carries information below the consonant–vowel level, mainly about manner. Figures and results will be added to appendix.

Figures R6 and R7 show cosine similarity by manner and by place of articulation.

Response to Critical Weakness 2:

Rotation in the brain. We agree, and we state this explicitly in Section 3.3 and in the Limitations. We note, however, that this concerns one component of a broader study. The core contribution of the paper is a mechanistic account of phoneme–position binding in neural models, established through causal interventions and baseline comparisons, which the reviewer finds strongly supports the Spiral hypothesis. The intracranial analysis then tests the prediction that the Onion and Spiral hypotheses share, a decrease of response magnitude with position, and confirms it in human STG, robustly to surprisal and acoustic controls. The rotational component is a new prediction that follows from our model results. Testing it requires many repetitions of the same phonemes at several within-word positions, which no existing dataset provides. We make it explicit so that future intracranial experiments can be designed to test it, and we see this as a contribution of the work rather than a gap.

Response to Question for authors:

Question on Figure 4B. The increase from position 6 to 7 is small (from 5.18 to 5.31) and well within the standard error. Position 7 contains only 8 tokens from five word types.

Reviewer_sCNeOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper tries to answer how artificial neural networks and human brains represent the position of phonemes when processing speech. It observes the internal states of LSTMs trained to repeat input phoneme sequences, in order to validate the "onion" or "spirial" hypotheses of phoneme position representations. They also analyzed human iEEG data to validate the hypotheses.

Critical strengths
  1. The research question is interesting.
Critical weaknesses
  1. I find the settings of the LSTM experiments inconvincing. The input and output of the LSTMs are the same phoneme sequences, which could actually be replaced with any token sequences, making the mechanism findings not unique for phonemes. Also, repeating phoneme symbols are far from modeling phonemes in natural speech or text (if the models do have such mechanisms), so the findings cannot generalize beyond this toy setting.
  2. The results on the LSTMs and human brains are weakly related. They are performed with different inputs, and the operations and results in these two scenarios cannot align. For example, the anti-factual intervention cannot be performed on the human brain, and in the human brain the norm analysis cannot distinguish the onion and the spiral hypothese.
Questions for authors

None

1 comment

We thank the reviewer for their time and for finding the research question interesting. Below we address the Critical Weaknesses raised by the reviewer.

1. LSTM setting. Binding an identity to a position is a general problem that any system processing sequences must solve, whatever its tokens. This is why the onion representation was first proposed and tested with random token sequences (Csordás et al., 2024). Our contribution is to test it on English words, where phonotactics constrain which phonemes can follow which. This natural structure revealed effects that random sequences cannot show. On English words, magnitude alone was not sufficient to encode position, and a rotation was needed (the spiral model). Intervention success also depended on whether the edited word is phonotactically legal (Table 4; Appendix C). The findings are therefore not those of arbitrary tokens: they depend on the statistics of the phoneme sequences. Repeating a sequence is also a standard task for studying role–filler binding in recurrent networks. McCoy et al. (2019) trained sequence-to-sequence RNNs to repeat digit sequences and showed that they implement tensor product representations, of which our onion and spiral models are compressed forms (Section 4). More generally, small models trained on controlled tasks are commonly used to isolate a mechanism and test it causally before studying larger systems (e.g., Elhage et al., 2022). We agree that testing whether the same mechanism emerges in models of natural speech is an important next step, and we will add this to the Limitations.

2. Relation between the model and the brain. We agree that the two settings differ. The model reads phoneme symbols and allows causal interventions, whereas the human recordings come from natural speech and allow only observational analyses. Such a gap is common when neural networks are used to study the brain. Saxe et al. (2021) and Doerig et al. (2023) argue that artificial networks serve neuroscience best as sources of explicit, falsifiable hypotheses about brain computation, to be tested against neural data, rather than as replicas of the brain. We follow this approach. We used the model to derive hypotheses about how position can be encoded, tested them causally in the model, and then tested in STG the predictions that the available recordings allow. STG responses show the decrease in magnitude with position that both hypotheses predict, robust to surprisal and acoustic controls. In addition, the direction of the response carries phoneme identity information, including below the consonant–vowel level (see our response to Reviewer 3). As the reviewer notes, the rotation that distinguishes the spiral from the onion model cannot be tested with these data. We state this in Section 3.3 and the Limitations, and we offer it as an explicit prediction for future intracranial studies, which would require repeated phonemes across positions. We will add a short paragraph at the start of Section 3.3 stating this logic, together with a table listing each prediction and where it is tested.

References

  • Csordás, R., Potts, C., Manning, C. D., & Geiger, A. (2024). Recurrent neural networks learn to store and generate sequences using non-linear representations. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP (pp. 248–262). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.blackboxnlp-1.17
  • Doerig, A., Sommers, R. P., Seeliger, K., Richards, B., Ismael, J., Lindsay, G. W., Kording, K. P., Konkle, T., van Gerven, M. A. J., Kriegeskorte, N., & Kietzmann, T. C. (2023). The neuroconnectionist research programme. Nature Reviews Neuroscience, 24(7), 431–450. https://doi.org/10.1038/s41583-023-00705-w
  • Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., Grosse, R., McCandlish, S., Kaplan, J., Amodei, D., Wattenberg, M., & Olah, C. (2022). Toy models of superposition. Transformer Circuits Thread. https://transformer-circuits.pub/2022/toy_model/index.html
  • McCoy, R. T., Linzen, T., Dunbar, E., & Smolensky, P. (2019, May 6–9). RNNs implicitly implement tensor-product representations [Paper presentation]. International Conference on Learning Representations, New Orleans, LA, United States. https://openreview.net/forum?id=BJx0sjC5FX
  • Saxe, A., Nelli, S., & Summerfield, C. (2021). If deep learning is the answer, what is the question? Nature Reviews Neuroscience, 22(1), 55–67. https://doi.org/10.1038/s41583-020-00395-8
Reviewer_tzQnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates how recurrent neural architectures bind phoneme identity (the filler) to its serial position within a word (the role). The paper argue that update magnitude encodes position while update direction encodes identity. To verify this hypothesis, the authors analyze a pretrained single-layer LSTM encoder-decoder trained to repeat English words, alongside human intracranial electroencephalography data from 9 participants. While the model's state norm indeed decays with position, the state update direction for a given phoneme drifts across positions, prompting the authors to propose the "spiral hypothesis" - a refinement that combines exponential magnitude decay with a rotary position embedding style rotation. Through causal additive state interventions evaluated via exact sequence match on a frozen decoder, the spiral model outperforms both the onion model. A crossed ablation attributes this performance advantage directly to the rotation mechanism, and state edits succeed significantly more often when target outputs conform to phonotactic constraints. On the human side, the study evaluates high-gamma activity across 59 superior temporal gyrus electrodes listening to natural spoken sentences. The neural response change between consecutive phonemes decreases monotonically with word position, a gradient that persists even when controlling for phoneme cohort surprisal. Neural difference vectors also display modest alignment within consonants and within vowels relative to across-class pairs.

Critical strengths
  • The computational evaluation is rigorously structured. By parameterizing each theoretical hypothesis into explicit state edit operators and testing them against frozen decoder readouts, the authors establish a principled intervention framework.

  • The paper provides a clear, falsifiable bridge between artificial neural network interpretability and human neuroscience.

Critical weaknesses
  • Based on my understanding, the human cortical evidence fails to isolate a true positional code. In the recurrent model, Δz\Delta z represents an additive update to a persistent state that must retain the entire word history until output generation. In contrast, STG high-gamma responses reflect local population activity averaged over the current phoneme's time window, which is primarily driven by immediate acoustic input. Because the paper does not demonstrate that STG activity maintains preceding phoneme history, a decreasing norm later in a word does not necessarily imply a positional encoding. Some confounds that comes to mind: lexical stress patterns in English, where word-initial syllables carry higher emphasis and acoustic energy, or word length biases, given that higher positional indices (5–7) originate solely from longer lexical items.

  • Unlike the neural network analyses, which incorporate untrained baseline controls, the brain analyses lack an acoustic or spectrotemporal null comparison. Computing ∥Δi∥\Vert{}\Delta_i\Vert{} directly on stimulus spectrograms or auditory model predictions would clarify whether the magnitude decline is driven by low-level acoustic properties. Also treating 1,110 phoneme tokens as independent observations, ignoring nested dependencies across 508 words and pseudo-population pooling across 9 patients without hierarchical or linear mixed-effects modeling certainly overstates the reported p-value.

  • The modeling conclusions are constrained by a narrow experimental footprint, relying on a single single-layer LSTM trained on English. Methodologically, the spiral form does not seem to be the optimal description of the model itself: low-rank and per-position operators achieve superior accuracy (0.780 and 0.773 vs. 0.729, Table 7), and the best structured edit still fails on approximately 28% of pseudoword targets (Table 1). The authors interpret the performance gains from phonotactic filtering as evidence that language statistics shape the positional code. But does this not demostrate that state edits succeed or fail depending on surrounding phonemic context—meaning the positional code is context-dependent?

  • A relevant work should be cited and discussed, Gwilliams et al. (2022, Nat. Commun.). Recording MEG from 21 participants listening to spoken narratives, Gwilliams et al. demonstrated that the auditory cortex holds approximately three recent phonemes in parallel using a dynamic, time-varying code that simultaneously represents phonetic content and elapsed time, with phonetic features remaining decodable across positions. Does this contradicts the paper's framing that prior work characterizes identity alone while leaving role-filler binding to position open?

Minor Weakness:

  • Claiming validation in "the human brain" and referencing "20 participants" in the title and abstract oversells the empirical scope, given that only 9 participants contributed usable STG electrodes.

  • typo: "ranodmlly" (Line 202).

Questions for authors
  • Does the positional magnitude decay hold consistently when evaluated individually within each of the 9 contributing participants?

  • Does the consonant/vowel alignment effect persist when phoneme pairs are strictly matched on the phonetic class of the preceding phoneme?

  • Why Fig 2B and Fig 4C have similar data structure but completly different figure style used? This makes it harder to compare between the two, this is also the case for other subfigures...

3 comments

We thank the reviewer for their time and their thorough, constructive feedback. Below, we provide a point-by-point response to each comment in the order presented:

Response to Critical Weakness 1:

Contextual Embeddings Eq. 1 and Eq. 6 are analogous ways to compute how the network state changes when a new phoneme arrives, in the RNN and in the STG population. In neither case do we know exactly how earlier phonemes are preserved, so both analyses use the change in the global population activity of the network To clarify this, we will make the following modifications to Section 2.3, "This captures the contribution of each successive phoneme" will become "As Δzi\Delta z_i in the model, Δi\Delta_i measures how the population state changes when a new phoneme arrives".

However, we agree that the two tasks – word repetition for the RNN and passive listening for humans – are not identical, and that the former may encourage the population state to better preserve information about previous phonemes. We will therefore add the following clarification to the Limitations section: "The two tasks – word repetition in the RNN and passive listening in humans – differ in their computational demands. Because word repetition requires holding an entire sequence in memory for production, it may encourage the RNN's population state to preserve information about earlier phonemes more strongly than passive listening does in humans."

Confounds. To address the point regarding possible confounds, we regressed ‖Δ‖ on position as well as potential confounds. If the prediction from onion-like encoding is correct, the coefficient of phoneme position should remain significantly negative, also when taking in consideration the confounds. The confounds we included are: surprisal, stress, phoneme duration, sound level and spectrogram Δ (Eq. 6 on the stimulus log-mel spectrogram). Standard errors are clustered by word, and β is standardized.

Predictorβp
Position−0.131.7×10⁻⁶
Surprisal0.030.31
Stressed vowel−0.290.019
Duration−0.361.5×10⁻¹⁶
Sound level0.211.3×10⁻⁶
Spectrogram Δ−0.114.9×10⁻⁴

Figure R1 shows the standardized β of each predictor with its confidence interval.

That is, position remains significant. We will add these regression results to section 3.3 and update Figure 4 accordingly.

Response to Critical Weakness 2:

Acoustic Model Comparison To test whether the decrease reflects the sound rather than the neural response, we applied Eq. (6) to the log-mel spectrogram of the stimulus, averaged over each phoneme interval. Unlike the neural ‖Δ‖ (ρ = −0.38, p = 1.6×10⁻³⁸), the spectrogram ‖Δ‖ does not decrease with position (ρ = −0.03, p = 0.32, n = 1110; Figure 4B).

Figure R2 shows ‖Δ‖ of the neural response and of the spectrogram by position, each relative to position 1 (shading is SEM).

We will add this control analysis to the new Appendix A.4 and add the following text to Section 3.3 "Applied to the stimulus log-mel spectrogram, Eq. (6) shows no decrease with position (ρ = −0.03, p = 0.32), so the decrease in neural ‖Δ‖ is not inherited from the acoustic input (Appendix A.4)."

Response to Critical Weakness 3:

Model generalization. Onion and Spiral are explicit models that define how positions and identities are bound together, whereas per-position and its low-rank variant learn a separate vector for each position and serve as an upper bound on accuracy (Appendix E.1). Unlike the latter, Onion and Spiral can be evaluated on unseen positions, and found to generalize well, as reported. Furthermore, beyond the unseen phoneme-position combinations already tested with pseudo-words, we left out one position in intervention training and tested it on all positions, which is, again, impossible for per-position and low-rank methods.

Figure R3 shows accuracy at each left-out position with Spiral model, averaged over three seeds.

To clarify that per-position and low-rank serve as upper bounds, we will add to section 3.2 the following sentence “Note that because Spiral and Onion are explicit models of position-identity binding, they can be evaluated on positions left out of training, and indeed generalize well with minimal loss, except for the first position (Appendix E.6). In contrast, per-position and low-rank scaling learn a separate vector for each position; as a result, they serve as an accuracy upper bound (Table 7) but cannot be applied to unseen positions” and the full results to a new Appendix E.6.

Response continued in the next comment.

(Continued from the previous comment.)

Phonotactics. “Our structured edit is context-free: it depends only on position and on the base and target phonemes, not on surrounding phonemes. That this single edit succeeds on 72% of pseudowords and 93% of real words suggests that a large part of the positional code is shared across contexts. Success is nevertheless context-dependent: failures are concentrated on targets that violate the phonotactics of the training language, and keeping only targets with attested trigrams raises accuracy to 87% (Table 4). This dependence may arise in the encoder, if the positional code varies with phonemic context, or in the decoder, which may reject edits that yield improbable sequences. Our results do not distinguish between these possibilities, which we leave for future work.”

We will also revise the Introduction to state that intervention success, rather than the positional code itself, depends on the statistics of the language.

Response to Critical Weakness 4:

Additional references. Gwilliams et al. (2022) show that a phoneme's neural pattern evolves with time since its onset, marking the relative order of recent phonemes, and that position in the word is decodable, but not how identity and position are combined in one population state. Their identity decoders generalize across positions but perform best when training and test positions match, and the authors leave open whether this reflects position-specific encoding. As they decode the neural response at successive time points rather than its change (Δ), this is not a direct test of our hypotheses, although generalization fits a shared identity direction, as in both hypotheses, and the matched-position advantage fits the rotation of the spiral (aligned with Figure 2C comparing similarity between position in contextualized embeddings). We propose a geometry for this combination, test it causally in a neural model and test its magnitude prediction in STG. A recent work by the same author (Gwilliams, 2026) proposes that rotational dynamics may solve the sequencing problem by time-stamping representations, so that their location on a neural trajectory changes systematically with elapsed processing time. In line with this, intracranial recordings show that STG population activity tracks relative elapsed time within words, along a trajectory from word onset to offset (Zhang et al., 2026). These findings are consistent with the rotation of the state proposed by our spiral hypothesis.

We agree that this work is relevant and will cite Gwilliams et al. (2022) together with these two recent studies. In the Introduction, "This prior work characterizes the encoding of identity alone, however, leaving open how it is bound to a position." will become "Gwilliams et al. (2022) showed with MEG that phonetic features remain decodable across positions in the word, and that each phoneme's neural pattern evolves with time since its onset, encoding the relative order of recent phonemes. In STG, population activity tracks relative elapsed time within a word (Zhang et al., 2026), and rotational dynamics of this kind have been proposed to time-stamp speech representations (Gwilliams, 2026). These studies show that content and elapsed time are encoded jointly, but not by what geometry identity is bound to its position in the word."

References

Response continued in the next comment.

(Continued from the previous comment.)

Response to Question for authors 1:

Participants. ρ is the Spearman correlation between position and ‖Δ‖ computed from each participant's electrodes only.(Figure R4).

With participants as the unit, the mean slope is negative (t(8) = −6.57, p = 1.7×10⁻⁴), as in a mixed-effects model with random intercepts and slopes (p = 4.9×10⁻¹¹). We will add per-participant results to Appendix A.4.

Response to Question for authors 2:

Previous phoneme class. As Δ subtracts the previous response, we also grouped pairs by the previous phoneme's class (mean cosine similarity).

Grouped byC-CV-VC-V
Current class0.0360.030−0.031
Previous class0.0140.062−0.027

Figure R5 shows cosine similarity by previous phoneme class.

Both groupings show within-class alignment but cannot be fully separated, because consonants and vowels mostly alternate (548 of 557 vowels follow a consonant), so 95% of pairs with two vowel predecessors are consonant pairs. Matched on a previous consonant, consonant and vowel pairs still align more than mixed pairs (AUC 0.57 and 0.58, p < 0.001). This result will be added to appendix figures.

Response to Question for authors 3:

The abstract will state 59 responsive electrodes from 9 participants, and Figures 2B and 4C will share the same style.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.