acceptodds
Under review as a conference paper at ICLR 2027

CREWS: Cortical Region Alignment for Enrollment Without Patient-Specific Parameters in Speech BCIs

Abstract

Intracortical speech brain-computer interfaces (BCIs) offer a promising pathway to restore communication for individuals with severe speech disorders. However, conventional systems are predominantly patient-dependent and suffer from the scarcity of large-scale paired training data for each individual. Although transferring a decoder trained on previous patients could reduce this burden, cross-patient generalization remains challenging because electrode array placements cover distinct electrode indices and lack spatial alignment across subjects. Existing approaches typically mitigate this spatial mismatch by learning target-patient-specific input mappings, which fundamentally precludes zero-shot decoding. To overcome this limitation, we propose CREWS, a speech BCI framework that aligns neural recordings by common cortical regions rather than discrete electrode indices. CREWS reallocates each array index to its corresponding anatomical region slot and processes these inputs using shared, region-specific encoders. This design enables neural recordings from a novel patient to be decoded by parameters pre-trained on identical cortical regions, eliminating the need for patient-specific calibration parameters. In a transfer learning setup where a target patient shares only a single cortical region with the training cohort, CREWS pioneers viable zero-shot decoding, successfully demonstrating that cross-patient alignment without adaptation is attainable. Furthermore, with as few as 5 labeled adaptation sentences, CREWS achieves a phoneme error rate (PER) of 68.9%, outperforming baseline cross-patient methods that rely on patient-specific input layers. These results demonstrate that anatomical cortical-region alignment enables cross-patient transfer without patient-specific parameters and can substantially reduce the calibration data required to deploy intracortical speech BCIs.

Primary area:
applications to neuroscience & cognitive science
Keywords:
speech brain-computer interface, intracortical speech decoding, cross-patient transfer, zero-shot decoding, few-shot adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_doFQOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Authors are introducing CREWS, a new approach to generalize speech BCI models across subject in a (quasi) zero-shot fashion. Instead of relying on subject-specific read-outs they train a region specific encoder which has parameters for each region of the broadmann atlas in which electrodes could be implanted in the pre-motor cortex for attempted speech task. They use neural recordings coming from BrainGate2 studies, namely the Card et al, Willet et al and part of the Kunz et al datasets, all of which carry 1 participant implanted with multiple Utah arrays, sometimes with the same region overlapping across participants (in particular area 6v).

They proceed evaluating the model by training it on Card and Willett data using the baseline backbone GRU proposed earlier but replacing the subject and session specific layers with their shared per region encoder, averaging the activity over each region, showing sparks of generalization across subjects measuring the PER on held-out T16 subject (and rotating them) with zero or up to 60 calibration trials. Their main claim is that this approach is able to generalize across subject.

Critical strengths

There are a couple of things that I liked a lot:

  1. The overall idea of CREWS, moving from subject specific to region specific across subject. Clear and simple intuition.

  2. The way authors use the neural data statistic to reorder channels within each array (and therefore region). I found it clever, a sort of functional alignment done based on ranking, cool.

  3. The problem they want to target itself, which is today one of the major limitation in deployment of effective speech BCI.

Critical weaknesses

While the idea of the paper is cool, there are some weaknesses that I'd like to list.

  1. The most obvious one, and I understood it's not fault of the authors, is the limited in scope analysis on new subject. Data are extremely limited, and in the end, there is only one subject avaialble for generalization and 1-2 for training, with just one region overlapping. Claims of the paper I think overstate the claims given the narrower scope of the current analysis. These are mostly all the publicly avaialble data of this kind, so I see where this come from, but still limits the results. At least, I would suggest authors to include data from Kunz et al 2025 T12 and T15 and T17 participants. In case of T12 and T15 these are the same subjects of the Willett and Card et al papers used for training, but in a different setting: imagery speech vs attempted speech. We already know that imagery and attempted speech exhibit similar structure in the neural data (but a motor intent direction) and showing generalization across the same regions (because of same participants) but different task will strengthen the results of the paper. T17 comes in a similar fashion (imagined single words) and could help substantiate the message if there is at least an array implanted in shared region.
  1. I am not convinced that CREWS is decoding phonetic content in the zero-shot setting, or at least the current evidence does not show it clearly. CTC is prone to degenerate solutions, and a PER of ~78% leaves a lot of room for non-phonetic explanations. The trial-shuffle chance level holds the decoded sequences fixed and permutes the references. It therefore controls for the overall distribution of output lengths and for predicting frequent phonemes, but not for the trial-wise correlation between output length and reference length. Trial duration correlates strongly with the number of phonemes and words in a sentence, so a decoder whose output length simply tracks trial duration would already beat this chance level without capturing phonetic structure. A substantial part of the gap to chance (91.4 to 77.7) could stem from this length information alone. The zero-shot claim I think should be toned down across the paper.

I acknowledge that the within-array electrode shuffle in Fig. 2 raises PER to chance, which partially argues against a pure length or energy explanation. However, that manipulation also changes the input statistics and may simply miscalibrate the decoder, so it is not conclusive. I would ask the authors for direct controls: a length matched chance level (permuting references only among sentences of similar length), a “length oracle” baseline that emits frequent phonemes at the correct length, a substitution/insertion/deletion breakdown, and the hypothesis-to-reference length ratio. On the closed 50-word vocabulary, word identification accuracy or WER with a 50-word language model would also be far more interpretable than PER. Unless these controls clearly support phonetic decoding, I would recommend that the zero-shot claims be substantially toned down throughout the paper, including the abstract and the contribution list.

  1. At the end of the day, qith the current cohort, only v6v is shared across all subjects, and every target-subject result is obtained through the v6v slot alone (T16 is decoded exclusively from its v6v array). The evidence therefore supports a narrower claim than the one made: that v6v can be modelled across subjects with a shared encoder. Nothing is shown about transfer through the other regions or about the multi-region slot design as such, and the authors acknowledge this in the limitations. The method combines two distinct ingredients: (i) the shared region-specific front-end with slot projections and averaging, and (ii) the label-free reordering of channels within each array by autocorrelation rank. The paper does not clearly separate their contributions.

The “v6v slot only” ablation in Table 4 is informative here. Restricting every patient to the v6v slot gives a zero-shot PER of 79.0, essentially the same as full CREWS (which was 77.7), and the difference appears only in the few shot regime. On the other hand, removing the autocorrelation ordering barely affects zero-shot PER (80.7) but severely impairs adaptation (50.1 vs. 23.5 at k = 60). Taken together, this suggests that the channel remapping may be the key ingredient and the multi-region front-end a secondary one. To disentangle the two, I would ask for: (a) the shared-input-layer baseline (all arrays, no region slots) with the autocorrelation reordering applied; (b) the patient-specific (Levin-style) baseline restricted to the v6v array, with and without reordering; and (c) a model trained only on the v6v arrays of T12 and/or T15 with reordering, applied to T16 zero-shot and after k-sentence calibration. If these variants match CREWS, the contribution of the shared multi-region encoder would be smaller than claimed. The value of the remapping procedure, however, would come out more clearly, and in my view it is the most interesting and reusable idea of the paper.

Questions for authors
  1. See weakness 1, could you add more data to showcase cross-task benefit of this approach?

  2. See weakness 2, could you provide example of decoded sentences at every level of K? Is the zero-shot merely matching the sequence length or it already capture some phonetic structure? Is it just predicting frequent phones? On the same line, since CTC might be unstable, would you be able to provide seed-by-seed variability of performance over K? the curve is not always monotic, even though they all converge stably at K=60.

  3. Please see weakness 3

  4. It's a bit unclear which data were used T16. It's attempted speech or imagined speech? Isolated words or full sentences? Have you checked for repetition/leakage of train/test data? Often sentences in Kunz dataset are repeated and a random split might allow for some unwanted leakage.

  1. What does the autocorrelation ordering actually align? The autocorrelation of threshold crossings and spike-band power at 40–100 ms lags is likely to covary strongly with firing rate and signal-to-noise ratio. The ordering may therefore sort electrodes by recording quality (“good channels first”) rather than by any functional correspondence across patients, and the shared encoder would then learn a quality-weighted readout. Could the authors compare the autocorrelation order against simple orderings by mean firing rate, SNR, or variance? If these yield similar zero-shot and few-shot PER, the mechanism behind the remapping would need to be reframed. Relatedly, how stable is the ranking within a subject across sessions (test-retest Spearman), and how much unlabeled data is needed before the order, and the downstream PER, stabilizes?

  2. Is there a shared latent geometry, or is the GRU compensating? You are arguing that region slots create corresponandance across patients, but it was never shown at representations levels. Could you compare the 6v encoder outputs across T12, T15 and T16 for matched sentences or words or phonemes? I would suggest to use RSA, CCA or Procrstes alignment testing with and without the autocorrelation ordering, versus the normal baseline read in layer. It would be nice to see whether the model is able to project different subjects in a common geometric space.

5 comments
AuthorsOpenReview ↗

Response to Reviewer doFQ (1/3): W1·Q1, W2, Q2

We thank the reviewer for the thoughtful assessment and feedback. We performed the requested experiments and respond point by point.

W1 · Q1 (data of Kunz et al., 2025; T17). As suggested, we evaluated CREWS on the data of Kunz et al. (2025), and these results will be added to the revision. None of the sentence evaluations used a model trained on imagined speech. For T17, whose data contain only isolated words, CREWS was adapted on 10 imagined trials per word.

EvaluationAdaptationCREWS PER (%)Chance (%)
T12 imagined sentences (100 trials)None74.1118.8
T15 imagined sentences, 50-word vocabulary (250 trials)None54.297.9
T15 imagined sentences, 125k vocabulary (270 trials)None64.296.1
T16, 50 new imagined sentences60 attempted sentences50.9102.1
T17, 7 imagined isolated words (70 trials)10 imagined trials per word82.697.1

In every evaluation of the table, the PER of CREWS is clearly below the chance level. These results indicate that CREWS also applies to other tasks. No evaluation used patient-specific parameters.

W2 (whether zero-shot decoding reflects phonetic content). We added all requested controls (General Response, G1), and these controls show that the zero-shot result is not explained by length.

The length-matched chance level (90.9%) and the length oracle (89.62%) both remain far above the PER of zero-shot CREWS (77.7%). The error breakdown and the hypothesis-to-reference length ratio are listed in the following table. Substitutions are the most frequent error, and the zero-shot output is neither empty nor far shorter than the reference sentence.

Error breakdown of zero-shot CREWSValue
Substitutions / deletions / insertions (% of reference phonemes)48.3 / 22.8 / 6.6
Hypothesis-to-reference length ratio0.84

Following the suggestion, the WER on the 50-word-vocabulary sentences of T16 was computed using a language model. The language model is the 3-gram released by Willett et al. (2023), and decoding used the same settings as the evaluation of T12. Word identification accuracy denotes the proportion of the words in the reference sentence that appear in the decoded output. The values in parentheses were obtained by permuting the reference sentences among trials of similar length.

CREWSWER % (permuted)Word identification accuracy % (permuted)
zero-shot90.0 (98.1)13.3 (9.0)
k=573.0 (97.4)29.2 (9.7)
k=1057.2 (95.9)45.9 (16.5)
k=2038.7 (97.3)64.5 (14.8)
k=6022.3 (96.2)80.9 (15.1)

The WER is below the permuted value already in the zero-shot setting, and the margin increases with the number of adaptation sentences.

Regarding the suggestion to tone down the zero-shot claim, the requested controls indicate that the zero-shot result reflects phonetic content rather than length. We therefore retain the claim that zero-shot decoding is below the chance level. However, we agree that the phrase "viable zero-shot decoding" in the abstract can be read as a claim of accuracy at a usable level. We will remove "viable" in the revision so that the abstract matches the result.

Q2 (decoded examples at every k, frequent phonemes, seed variability). Decoded examples at every k for three evaluation sentences are shown in the following table and will be included in the appendix of the revision.

k"You are okay.""I do good.""You do not need faith."
0up dohopedo
5how are theyhello you gooddo you they
10you are theythey do hereyou need that feel need
20you are no theyfaith do goodthey do not have need faith
60you are hope theyI do goodyou do not need faith

The zero-shot output is not restricted to frequent phonemes, as the token-type count and the length oracle in the General Response (G1) show.

Across three seeds, the standard deviation of CREWS lies between 1.2% and 1.5% at every k. For comparison, the following table lists two baselines that add a new patient-specific input layer for T16 to a source model. The frozen decoder (Table 1) trains only this new layer. The fine-tuned baseline, which is not part of Table 1, additionally updates all other parameters, using only the k sentences of T16. Both baselines yield a higher PER and a larger standard deviation than CREWS at every k.

Method (PER, %)k=5k=10k=20k=60
CREWS68.9 ± 1.361.5 ± 1.245.2 ± 1.323.5 ± 1.5
Patient-specific, fine-tuned86.3 ± 9.275.9 ± 3.870.6 ± 3.453.9 ± 10.7
Frozen decoder88.7 ± 3.987.1 ± 1.984.8 ± 6.483.6 ± 9.0
AuthorsOpenReview ↗

Response to Reviewer doFQ (2/3): W3·Q3, Q4

W3 · Q3 (separating the autocorrelation ordering from the region front-end). We trained the three requested variants (a)–(c) and added a model that uses only the v6v slot without the autocorrelation ordering as a control. CREWS with and without the autocorrelation ordering is listed in the first two rows for comparison.

Variant (PER)InputOrderingZero-shot (chance)k=5k=10k=20k=60
CREWSall region slotsyes77.7 (91.4)68.961.545.223.5
CREWS, no orderingall region slotsno80.777.472.367.050.1
(a)shared input layeryes99.6 (99.5)74.857.146.228.0
(b)patient-specific layer, v6veither–71.968.970.825.1
(c)v6v slot onlyyes79.077.369.157.428.4
Controlv6v slot onlyno79.1 (96.9)78.367.267.866.2

Ordering denotes the autocorrelation ordering. The first, second, and (c) rows are from Tables 1 and 4. The zero-shot entry of (b) is empty because the model produces no output without T16 labels.

(a) On T16, the zero-shot PER of the shared input layer is at the chance level with or without the autocorrelation ordering. Zero-shot transfer therefore requires the region slots. In adaptation, variant (a) attains a lower PER than the shared input layer without the autocorrelation ordering at k=5, 10, and 20 (Table 1). As suggested, the autocorrelation ordering can therefore be reused independently, although its benefit is smaller without region slots.

(b) For the patient-specific input layer, the model with the autocorrelation ordering and the model without the autocorrelation ordering are the same model. The autocorrelation ordering only permutes the columns of that layer, and the logits of the two models differ by less than 1e-7.

(c) With only the v6v slot, removing the autocorrelation ordering leaves the zero-shot PER almost unchanged, whereas adaptation degrades at k=20 and k=60 (rows (c) and Control). CREWS exhibits the same pattern, as the first two rows of the table show.

In summary, the region slots account for zero-shot transfer and the autocorrelation ordering accounts for adaptation. The effect of the autocorrelation ordering is absent for patient-specific input layers, small for the shared input layer, and largest in combination with the region slots. Neither component substitutes for the other, and this distinction will be stated in Sec. 5.4 of the revision.

Q4 (T16 data). The T16 data are attempted-speech sentences from a single session. No sentence is repeated, and the overlap between adaptation and evaluation sentences is 0. The zero-shot PER of CREWS is unchanged on the 75 sentences whose transcript does not appear in the T15 training set (table in the General Response, G2). The Setup of the submission did not describe these conditions in sufficient detail. The Setup of the revision will state the task (attempted speech, sentences), the number of sessions, and the absence of repeated sentences, and the appendix will report the overlap with the T15 training sentences.

AuthorsOpenReview ↗

Response to Reviewer doFQ (3/3): Q5, Q6

Q5 (what the autocorrelation order aligns). To examine whether the order ranks electrodes by recording quality, we replaced only the ordering key with the firing rate, the variance, or the SNR and repeated training from the source stage.

Training with a different ordering key (PER, %)Zero-shotk=5k=10k=20k=60
Firing rate83.580.174.347.426.3
Variance90.475.565.053.860.5
SNR82.977.256.542.327.8
Autocorrelation (CREWS, Table 1)77.768.961.545.223.5

The autocorrelation ordering yields the lowest PER in the zero-shot setting and at k=5 and k=60. The autocorrelation ordering therefore captures more than a ranking by recording quality.

We also examined the stability of the order across recording days and the amount of unlabeled data that the order requires. The order of T16 was computed without labels from its 60 training trials, which were recorded in a single session. Reproducibility across recording days is therefore reported for T15 and T12. Across recording days, the median Spearman correlation of the v6v order is 0.68 for T12 and 0.31 for T15. The order can therefore differ across recording days, and this point will be stated in the Limitations of the revision. The required amount of unlabeled data is reported in the following table. For the downstream PER, CREWS was adapted on k sentences of T16 with the order computed from the same k trials.

Number of unlabeled trials510205060
T15, Spearman with the order from all trials (v6v)0.690.750.870.930.94
T12, same measure0.880.920.960.980.98
T16, Spearman with the order from the 60 training trials0.940.970.991.001.00
T16 downstream PER (%)77.972.047.131.823.5
CREWS, order from the 60 training trials68.961.545.230.723.5

For T12 and T16, the order itself converges with 20 unlabeled trials (ρ ≥ 0.96 with the reference order). For T16, the downstream PER follows a similar trend: with 20 or more trials, the order from the k adaptation trials yields a PER close to that of the order from the 60 training trials.

Q6 (shared latent geometry). Following the suggestion, we compared the output of the v6v region encoder of the three patients using phoneme-level RSA and Procrustes alignment. Top-1 denotes the proportion of phonemes whose nearest prototype in the other patient, after Procrustes alignment, is the same phoneme (chance approximately 0.03).

v6v region encoder outputRSA T15–T12RSA T16–T15RSA T16–T12Top-1 Procrustes T15–T12
CREWS0.410.210.180.28
No electrode autocorrelation ordering0.220.110.060.11
Shared input layer0.250.090.010.08

The region encoder outputs of CREWS share the phoneme similarity structure across all three patient pairs, including T16, which was not used for training. All four values are higher than those of the model without the autocorrelation ordering and of the shared input layer, indicating that the autocorrelation ordering strengthens this shared structure when combined with the region slots. A structure shared across patients is therefore already present at the region encoder output, before the GRU. This analysis will be included in the appendix of the revision.

We are grateful for the detailed suggestions, which led to most of the new analyses in the revision. We would be glad to provide any further control that would be useful.

Reviewer_ZnNLOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes CREWS, a cross-patient intracortical speech BCI framework that aligns neural recordings according to cortical regions. Each cortical region is assigned a shared slot, processed by a region-specific encoder and projection, and the available slots are averaged before being passed to a shared speech decoder. Coverage dropout is used during training so that the model can operate when only a subset of cortical regions is available.

The paper's main claim is that such anatomical alignment enables zero-shot speech decoding on unseen patients without patient-specific learned parameters and further adaptive training. In the zero-shot setting, CREWS obtains 77.7% PER compared with a reported chance level of 91.4%. However, with adaptive training, PER decreases to 23.5% using 60 sentences. Such a pronounced performance disparity may still address the necessity of subject-specific fine-tuning, thereby undermining the authors' claims regarding zero-shot feasibility.

Critical strengths
  1. This paper addresses an important topic. Subject-specific adaptation is a major practical challenge for intracortical speech BCIs. This paper proposes a zero-shot approach above chance level. A method that could genuinely generalize across subjects without specific learned parameters would be highly impactful.
  2. The paper is clearly written and easy to follow, and most of the methods are reasonable. The motivation for processes such as replacing electrode-index with cortical-region correspondence is intuitive. The architecture is simple and well explained, and the experimental protocol is generally presented clearly.
  3. The authors construct detailed ablation studies and hold-out experiments. Ablation studies demonstrate that replacing region alignment with a shared input layer or removing coverage dropout increases zero-shot PER (although this conclusion aligns fairly well with intuition). For held-out settings, the paper additionally evaluates T15 and T12 as target patients. CREWS remains better than the reported baselines, making the experiment results more stable.
Critical weaknesses
  1. The paper does not convincingly establish a general cross-patient cortical representation. The entire study contains only three patients. More importantly, v6v is the only cortical region shared across all three patients. Consequently, the main zero-shot experiment effectively reduces to transferring a representation learned from the v6v arrays of the source subjects to the v6v array of the target subject. This is considerably narrower than the paper's broader framing of patient-independent cortical-region alignment. The ablation results make this issue more concerning. A model using only the v6v slot obtains 79.0% zero-shot PER, compared with 77.7% for full CREWS. The difference is quite small.
  2. A simpler cohort-average explanation has not been ruled out. The current results are also compatible with a simpler interpretation, that is, at a high level, CREWS pools recordings from a very small number of source subjects, maps corresponding anatomical regions into a common representation, and averages the available region representations before decoding. For the main zero-shot experiment, the target subject ultimately contributes only the v6v region. CREWS may not literally compute the arithmetic mean of three patients' neural signals. Rather, with only two source patients in the main experiment, the shared v6v encoder may simply learn a representation that lies somewhere between the source subjects and therefore happens to be slightly closer to the unseen patient's distribution than an unaligned input representation. The small number of patients makes this alternative explanation particularly difficult to exclude.
  3. The adaptation results actually strengthen the concern above. The zero-shot result is the main conceptual novelty of the paper, but its absolute performance is extremely weak. For T16: zero-shot: 77.7% PER; 5 labeled sentences: 68.9%; 60 labeled sentences: 23.5%. The pretrained cross-subject model may simply place the target subject in a somewhat better starting region of parameter space than chance or an unaligned baseline. Once even a small amount of target-specific supervision is introduced, performance improves substantially, and the improvement is dramatic with more target data. Under this interpretation, the 77.7% zero-shot PER does not demonstrate that the model has learned a robust subject-independent speech representation. Instead, it demonstrates that the source cohort provides a weak but non-random prior. This distinction is important because the paper's novelty rests heavily on the zero-shot claim.
  4. The zero-shot result is statistically detectable but practically extremely weak. The paper emphasizes that 77.7% PER is below the 91.4% shuffled chance level. However, the absolute performance is still extremely poor. It is very far from showing that speech can meaningfully be decoded from a previously unseen patient without calibration.
  5. The method itself is conceptually simple relative to the strength of the claimed contribution. All of the model processes are reasonable choices. However, none appears technically deep or particularly novel by itself.
Questions for authors
  1. Only v6v is shared across all three patients, and the v6v-only ablation achieves 79.0% zero-shot PER, close to the 77.7% of the full CREWS model. Given that the main transfer experiment effectively relies on the same cortical region across source and target patients, how do the authors establish that the proposed multi-region alignment framework provides a meaningful advantage beyond learning a shared representation for v6v?
  2. What evidence demonstrates that CREWS learns a genuinely patient-invariant neural representation rather than a weak shared prior or central tendency over the source patients?
  3. Ho much useful information is actually transferred zero-shot? Beyond showing statistical significance against a shuffled chance level, can the authors provide a more interpretable measure of the amount of transferable speech information? or example, phoneme-level mutual information, decoding accuracy on restricted phoneme sets, or another meaningful analysis.
  4. How should the adaptation curve be interpreted? Performance improves from 77.7% zero-shot PER to 68.9% with 5 labeled sentences, 61.5% with 10, 45.2% with 20, and 23.5% with 60. This suggests that the source-patient model primarily provides a weak initialization and that most useful information remains target-patient specific. How do the authors distinguish their claimed patient-invariant representation from a standard transfer-learning prior that becomes useful only after target adaptation?
2 comments
AuthorsOpenReview ↗

Response to Reviewer ZnNL (1/2): W1·Q1, W2·Q2, Q3, W3·W4·Q4

We thank the reviewer for recognizing the importance of the problem, the clarity of the presentation, and the ablation and hold-out experiments. We respond point by point.

W1 · Q1 (only v6v is shared; advantage of the multi-region design). Transfer is not limited to v6v. As described in the General Response (G3), T15 and T12 were decoded through both shared slots, v6v and d6v, when either was the held-out target patient. The Setup of the revision will state this point explicitly, and the Results and the Conclusion will state the regions used for each target patient. The advantage of the multi-region design also appears in two other results.

(1) Restricting every patient to the v6v slot leaves the zero-shot PER similar but degrades adaptation at every k (the "v6v slot only" row and the CREWS row of Table 4). A decoder trained jointly on arrays from multiple regions therefore provides a better starting point for adapting to a new patient.

(2) Patients with different region coverage are combined in one model without loss of accuracy. On the source patients, CREWS yields a PER comparable to that of a model with a patient-specific input layer (source columns of Table 2).

The architecture assigns one slot to each region, and a patient who shares a region other than v6v can therefore be added in the same manner, extending transfer to that region. In all of these experiments, transfer occurs without a patient-specific input layer.

W2 · Q2 (cohort average; patient-invariant representation). We agree that the zero-shot result reflects a representation shared across the source patients rather than a fully patient-invariant representation. A new patient is nevertheless decoded from that shared representation alone, without patient-specific parameters. Three results indicate that this shared representation does not arise merely from pooling the source patients.

(1) On T16, with the same source patients and the same decoder, a shared input layer yields a zero-shot PER of 91.6%, at the chance level (93.0%), whereas the region slots yield 77.7% (Table 1).

(2) The shared representation is specific to the region. Placing the v6v array of T16 in another slot, or another array in the v6v slot, raises the PER to 96–118% (Fig. 2).

(3) The phoneme similarity structure of the v6v region encoder output is shared across patients (RSA: T15–T12 0.41, T16–T15 0.21, T16–T12 0.18; table in the response to Reviewer doFQ, Q6). The corresponding values for the shared input layer are 0.25, 0.09, and 0.01.

Q3 (amount of transferred information). We computed two of the suggested measures: the phoneme mutual information and the PER on a restricted set of six manner-of-articulation classes, obtained by grouping the phonemes. The values in parentheses were obtained by permuting the reference sentences among trials of similar length.

CREWSManner-class PER, % (permuted)Phoneme MI, bits (permuted)
zero-shot63.8 (88.1)1.14 (0.29)
k=556.9 (88.5)2.27 (0.36)
k=6018.9 (86.7)3.29 (0.72)
Shared input layer, zero-shot89.1 (89.0)0.23 (0.22)

In the zero-shot setting, CREWS is better than the permuted value on both measures, whereas the shared input layer is almost identical to the permuted value. The transferred information increases with the number of adaptation sentences. This table will be included in the appendix of the revision.

W3 · W4 · Q4 (interpretation of the adaptation curve). We agree that the zero-shot result provides a non-random prior. This prior differs from a standard transfer-learning prior in three respects.

(1) In prior work, this prior was absent rather than weak. Levin et al. (2026) reported that transfer failed without a patient-specific input layer for the same BrainGate patients and concluded that such a layer is necessary for transfer.

(2) This prior arises from region alignment rather than from pooling the source patients. As shown in (1) and (2) of W2 · Q2, the same source patients and the same decoder yield a zero-shot PER below the chance level only when the region slots are used and each array is assigned to the slot of its region.

(3) The difference appears before adaptation. Methods with a patient-specific input layer produce no output without labels from the target patient ("–" in Table 1), whereas CREWS yields a PER below the chance level without labels. The caption of Table 1 in the revision will state the meaning of "–".

The adaptation curve therefore shows that adaptation starts without new parameters. Training only the input layer with the source decoder frozen remains at a PER of 83.6% even with 60 sentences (Table 1; CREWS 23.5%). However, the phrase "viable zero-shot decoding" in the abstract can be read as accuracy at a usable level. We will remove "viable" in the revision so that the phrase matches the result.

AuthorsOpenReview ↗

Response to Reviewer ZnNL (2/2): W5

W5 (simplicity of the method). We appreciate this comment and would like to clarify the rationale of the design. The architecture of CREWS follows from its purpose. Removing patient-specific parameters requires an input defined in units that correspond across patients rather than in units of one patient, and we selected the cortical region as that unit. Each component corresponds to one requirement of this design. The region slots provide the correspondence across patients, and the average over the present slots handles region coverage that differs across patients. Coverage dropout handles a new patient who has only a subset of the regions, and the autocorrelation ordering provides the correspondence within an array. The contribution of this paper is that this architecture removes the patient-specific input layer that prior work concluded to be necessary.

We thank the reviewer again for the critical reading. We hope that the additional evidence clarifies the scope of the claim, and we remain available for further discussion.

Reviewer_TcEHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

I am not an expert in speech decoders

The goal here is to align information across intracortical recordings by the brain area. The hope is to allow zero shot performance by understanding the decoding by the brain area. Of course, it makes a lot of sense to pay attention to the recorded brain area (the cited Barista work takes this approach). The claim is that they achieve zero shot performance better than chance, which is true but barely so.

Critical strengths

The approach does achieve better than change zero shot decoding

The work uses ablations nicely and seems to provide an empirical understanding of floor (chance) behavior

The paper is very honest about its limitations and does not generally over claim results

Critical weaknesses

There may be phoneme length dependent (instead of identity) effects. The permuations/shuffles need to be done matching length. This may explain the above chance effect

The bottom line is that decoding is far worse than the single patient trained decoders so it is hard to get too excited

it is not clear that there is a true independent validation set

Questions for authors

The authors need to control for phoneme length

Can a true and larger validation set be used?

What are the true implications of achieving a barely better than chance universal decoder?

1 comment
AuthorsOpenReview ↗

Response to Reviewer TcEH

We thank the reviewer for the positive assessment of the ablations, the analysis of the chance level, and the statement of limitations. We respond to the three questions in order.

W1 · Q1 (control for phoneme length). We agree that reporting only the chance level across all trials leaves the role of length open. We added a chance level in which the reference sentences are permuted only among trials of similar length. This length-matched chance level is 90.9%, almost identical to the chance level across all trials (91.4%), and the zero-shot PER of CREWS (77.7%) is clearly lower. The zero-shot result therefore does not arise from sentence length. The table and the additional controls are reported in the General Response, G1.

W3 · Q2 (independent and larger validation set). Zero-shot evaluation uses no T16 label, and all 120 trials constitute the evaluation set. In adaptation, the adaptation sentences and the evaluation sentences do not overlap, and all results are the values at the last training epoch (300 epochs for source training, 100 epochs for adaptation). The Setup of the revision will state that the zero-shot PER was computed on all 120 trials and that the results are last-epoch values.

Regarding a larger validation set, the attempted-speech data of T16 contain only these 120 sentences, and the set cannot be extended within the same task. Instead, we evaluated on 50 new imagined-speech sentences that were not used for adaptation. CREWS adapted on 60 sentences decodes these sentences with a PER of 50.9% (chance 102.1%).

W2 · Q3 (comparison with single-patient decoders; implications). The comparison with single-patient decoders depends on the amount of data, and the two cases are considered separately.

(1) With the same number of sentences from the new patient: a single-patient decoder trained only on k sentences of T16 yields a PER of 91.7–92.9% for k=5–60, whereas the PER of CREWS decreases from 68.9% to 23.5% (Table 1).

(2) With sufficient sentences: on the source patients, for whom thousands of sentences are available, CREWS attains accuracy comparable to that of a model with a patient-specific input layer (source columns of Table 2: T15 17.0% / T12 21.7%; patient-specific input layer 16.8% / 22.1%).

Single-patient decoders attain high accuracy only after thousands of sentences have been collected for each patient. This paper concerns the stage before that collection. In this stage, transfer occurs without labels in a setting where transfer without a patient-specific input layer was reported to fail (Levin et al., 2026), and the new patient attains a PER of 23.5% with 60 sentences and no new parameters. Because the zero-shot result is not a claim of accuracy at a usable level, we will remove "viable" from the phrase "viable zero-shot decoding" in the abstract.

We hope that these clarifications address the three questions, and we are happy to elaborate on any point.

Comments on the paper

3 comments

Dear Reviewer,

Thank you very much for your time and thoughtful review. As the discussion deadline approaches, could you please take a look at the authors' rebuttal and share your thoughts in the discussion?

Many thanks for your continued help!

AuthorsOpenReview ↗

General Response (1/2)

We thank the three reviewers for their thoughtful evaluations and constructive feedback. The core objective of this work is to generalize to unseen patients without introducing patient-specific parameters. Previously, Levin et al. (2026) evaluated the same BrainGate cohort and concluded that a patient-specific input layer is necessary for cross-patient transfer, as performance collapsed without it. In contrast, CREWS eliminates this patient-specific layer, achieves better than chance zero-shot decoding on unseen patients without target labels, and adapts rapidly using only a small number of sentences.

In response to the reviewers' suggestions, we have conducted all requested control experiments and analyses. Below, we first address the three shared concerns raised by multiple reviewers, followed by individual responses. Unless otherwise noted, all newly reported results are averaged across three random seeds.

G1. Zero-shot result and sentence length (TcEH W1 · Q1, doFQ W2, ZnNL Q3)

In the original submission, we reported a chance level computed by permuting the reference sentences across all trials. We agree that this global permutation does not rule out sentence-length effects as a contributor to the margin over chance. To control for this, we added a length-matched chance level, in which the reference sentences are permuted strictly within quintiles of reference length (evaluated across 120 T16 trials).

Zero-shotPER (%)Chance, all trials (%)Chance, length-matched (%)
CREWS77.791.490.9
Shared input layer (baseline of Table 1, no region slots)91.693.093.4

Controlling for sentence length leaves the chance level virtually unchanged, and zero-shot CREWS remains clearly below the length-matched chance level. Thus, our zero-shot performance is not driven by sentence-length cues. Two additional analyses further confirm this:

(1) Ground-truth length alone is insufficient: A length oracle that emits the most frequent phoneme matched to the exact reference length achieves a PER of 89.62%, well above the 77.7% PER of zero-shot CREWS.

(2) Diverse phonemic output rather than mode collapse: Zero-shot CREWS produces an average of 29.3 out of the 34 reference token types (phonemes and the word boundary), whereas the shared input layer collapses to only 6 token types.

We applied the same length-matched control to the adapted models. Across all values of k, CREWS achieves a PER substantially below the length-matched chance level, with this margin widening monotonically as the number of adaptation sentences increases.

CREWSPER, Table 1 (%)Chance, length-matched (%)
zero-shot77.790.9
k=568.988.9
k=1061.588.8
k=2045.287.5
k=6023.587.9

G2. Evaluation set and leakage (TcEH W3 · Q2, doFQ Q4)

The T16 dataset comprises 120 unique attempted-speech sentences recorded within a single session. Because zero-shot decoding requires no target labels from T16, we evaluated zero-shot PER across all 120 trials. For few-shot adaptation, we partitioned the data into two disjoint splits of 60 sentences each, drawing k adaptation sentences from one split and evaluating on the held-out 60 sentences, ensuring zero sentence overlap at every k. All reported metrics reflect the final training checkpoint (300 epochs for source training and 100 epochs for adaptation) without post-hoc checkpoint selection.

We also examined lexical overlap between the T16 evaluation sentences and the source training set. Of the 120 T16 sentences, 45 share an identical transcript with a T15 training sentence (though recorded from a different patient with distinct neural signals). Our zero-shot performance is not driven by this transcript overlap: on the 75 non-overlapping sentences, CREWS achieves a PER comparable to that on the 45 overlapping sentences and remains clearly lower than the shared input layer.

Zero-shot45 overlapping sentences: PER (%)75 non-overlapping sentences: PER (%)
CREWS76.978.1
Shared input layer91.291.9
AuthorsOpenReview ↗

General Response (2/2)

G3. Roles of the region slots and the electrode autocorrelation ordering (ZnNL W1 · Q1, doFQ W3 · Q3)

Because v6v is the sole recording region common to all three patients, cross-patient transfer to T16 was evaluated through v6v. Our statement in the manuscript that the patients share only v6v was intended in this three-way intersection sense. In fact, T12 and T15 also share d6v; accordingly, in our zero-shot experiments where either T15 or T12 served as the held-out target patient, decoding utilized both shared region slots (v6v and d6v). Because our architecture assigns a dedicated slot to each cortical region, any new patient sharing an arbitrary subset of regions can be integrated in the exact same way. Finally, we conducted the requested ablation/separation experiments, demonstrating that the region slots and the autocorrelation ordering play distinct functional roles (see the table in our response to Reviewer doFQ, W3 · Q3).

(1) Zero-shot transfer is driven by the region slots rather than the autocorrelation ordering. On T16, pairing a shared input layer with the autocorrelation ordering, but omitting region slots, collapses to the chance level (PER 99.6% vs. chance 99.5%). In contrast, retaining region slots while removing the autocorrelation ordering preserves nearly all zero-shot accuracy. CREWS without the autocorrelation ordering achieves a PER of 80.7% (Table 4), compared with 77.7% for CREWS.

(2) Few-shot adaptation is driven by the autocorrelation ordering. Removing the autocorrelation ordering alone from CREWS degrades the PER at k=60 from 23.5% to 50.1% (Table 4). Thus, region slots govern zero-shot transfer, whereas the autocorrelation ordering enables rapid adaptation.

Changes in the revision

  1. Setup: definition of the length-matched chance level. Appendix: the length controls (G1; doFQ W2), decoded examples at every k, and the WER with a 3-gram language model.
  2. Sec. 5.4: distinction between the roles of the region slots and the autocorrelation ordering (G3). Appendix: table of the separation experiments.
  3. Setup: T16 data conditions, zero-shot evaluation on all 120 trials, and last-epoch reporting (G2). Appendix: sentence overlap with the T15 training set (G2). Statement that both v6v and d6v were used when T15 or T12 was the target patient, and clarification in the Results and the Conclusion that transfer to T16 uses only v6v, whereas transfer to T15 or T12 uses both v6v and d6v (G3). Caption of Table 1: meaning of "–".
  4. Appendix: results for each seed, comparison of ordering keys, representation analysis, and evaluation on other tasks (imagined speech, T17 adaptation). Sec. 5.3 and Sec. 5.5 state the results on imagined speech and the representation analysis, and the caption of Table 1 reports the length-matched chance level.
  5. Limitations: reproducibility of the autocorrelation ordering across recording days (doFQ Q5).
  6. Abstract: "viable" removed from "viable zero-shot decoding".

We hope that this response and the revision resolve the concerns raised, and we welcome any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.