acceptodds
Under review as a conference paper at ICLR 2027

UniBCI: Towards a Unified Pretrained Model for Invasive Brain-Computer Interfaces

Abstract

Modeling invasive neural spike data is fundamental to advancing high-performance brain-computer interfaces (BCIs). However, existing approaches face critical challenges, including limited-scale heterogeneous data, cross-conditions distribution shift, and the intrinsic spatiotemporal complexity of invasive neural signals. In this work, we construct a pretraining corpus spanning multiple species, subjects, brain regions, and behavioral experiment paradigms. These heterogeneous recordings are standardized via our proposed unified normalization and tokenization. We also propose UniBCI, a unified pretrained model for invasive Brain-Computer Interfaces. The model integrates three key components: (1) a context-conditioned spatio-temporal tokenization (CST) scheme that embeds neural signals together with metadata into a shared representation space; (2) a hierarchical Interval-Area Attention (IAA) mechanism that captures patterns of spike dynamics in slots via linear attention and locality dependencies via sliding-window attention; and (3) a self-supervised masked signals reconstruction objective for rebuilding spike trains from unmasked neural tokens. We demonstrate that UniBCI outperforms baselines trained without behavioral data across both classification and regression tasks, while achieving performance comparable to leading behavior-supervised methods with fewer trainable parameters and lower inference latency. These results suggest that UniBCI provides a potential step toward general-purpose neural foundation models, enabling robust, scalable, and generalizable representation learning for invasive neural data. The code for this paper is available at: https://anonymous.4open.science/r/UniBCI-C805.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Invasive Neural Signals, Neural Decoding, Context-Conditioned Neural Modeling, Spatiotemporal Modeling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ay8QOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is proposing a unified framework designed to process heterogeneous invasive spike signals by: 1- Reconstructing masked spike tokens through a Context-conditioned Spatio-temporal Tokenization (CST) module to handle dataset variance across species, subjects, tasks, and sessions. 2- Capturing the spatio-temporal dependencies across local intervals and spatial brain areas using the Interval-Area Attention (IAA) block. 3- Capturing the general neural representations from unlabeled heterogeneous data byself-supervised masked spike reconstruction.

Critical strengths

The proposed framework was evaluated across various datasets for both classification and regression tasks. Additionally, extensive experiments were conducted to benchmark the approach against existing baseline methods. The results demonstrate that the model maintains high efficiency with a compact architecture, making it suitable for real-time applications.

Critical weaknesses

Mismatching in the dimension in the method equations which make it difficult to follow.

Questions for authors

1- Specify the dimension of Xemb∈RA×Tnorm×dX_{\text{emb}} \in \mathbb{R}^{A \times T_{\text{norm}} \times d} (or RTnorm×A×d\mathbb{R}^{T_{\text{norm}} \times A \times d}): While element-wise broadcast addition (⊕\oplus) works for TposT_{\text{pos}} and AposA_{\text{pos}} against a 3D tensor of shape (A,Tnorm,d)(A, T_{\text{norm}}, d), XembX_{\text{emb}} is described in the text as outputting token∈RTnorm×A×d\text{token} \in \mathbb{R}^{T_{\text{norm}} \times A \times d}. The ordering of axes between XembX_{\text{emb}} (RA×Tnorm×d\mathbb{R}^{A \times T_{\text{norm}} \times d}) and token\text{token} (RTnorm×A×d\mathbb{R}^{T_{\text{norm}} \times A \times d}) is flipped, causing dimension inconsistency.

2- Specify whether positional embeddings (TposT_{\text{pos}}) are applied before or after interval partitioning, as applying global positional embeddings prior to local interval-area attention can over-constrain local temporal dynamics (Eq 5)

3- In area-wise sliding window attention (Eq 9) : is additive masking ( m∈0,−∞m \in \\{0, -\infty\\} ) or multiplicative masking ( m∈1,0m \in \\{1, 0\\} )?

1 comment

(1) We thank the reviewer for pointing out the inconsistency in the tensor dimensions and axis ordering. We have corrected the notation of Xemb∈RTnorm×A×dX_{\mathrm{emb}} \in \mathbb{R}^{T_{\mathrm{norm}} \times A \times d} and the corresponding token representation in the manuscript to use a consistent axis ordering. We have also clarified how the element-wise broadcast addition with Tpos∈RTnorm×1×dT_\mathrm{pos} \in \mathbb{R}^{T_{\mathrm{norm}} \times 1 \times d} and Apos∈R1×A×dA_\mathrm{pos} \in \mathbb{R}^{1 \times A\times d} is performed, ensuring that the positional embeddings are added along their intended axes.

(2) We appreciate the reviewer's observation regarding the potential interaction between global positional embeddings and local interval-area attention. In our implementation, TposT_{\mathrm{pos}} is applied before interval partitioning. We have clarified this operation order in the method description and revised Eq. (5) accordingly. This operation makes the relationship between global temporal positions and local interval-area attention explicit.

(3) Thank you for pointing out the ambiguity in Eq. (9). In our implementation, the sliding-window mask is additive. Specifically, invalid attention positions are assigned −∞-\infty before the softmax operation. The binary mask M∈0,1\mathcal{M} \in \\{0,1\\} is for masking signal tokens.

We thank the reviewer again for these constructive comments. These revisions improve the mathematical precision and reproducibility of the method description.

Reviewer_GgAYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose UniBCI, a unified self-supervised pretraining framework for heterogeneous invasive neural spike signals, aiming to address data heterogeneity and distribution shifts across species, subjects, brain regions, and experimental paradigms, with downstream fine-tuning for specific BCI decoding tasks. The framework comprises data standardization, context-conditioned spatiotemporal tokenization (CST), hierarchical Interval-Area Attention (IAA), and neural representation learning. CST and IAA constitute the two core architectural components, combined with masked signal reconstruction for self-supervised pretraining. Experimental results show that UniBCI outperforms most baselines overall while maintaining fewer trainable parameters and lower inference latency, demonstrating its potential for cross-dataset transfer and efficient neural decoding.

Critical strengths

None

Critical weaknesses

1.The authors present CST, IAA, and Masked Signals Reconstruction as their main technical contributions, although their underlying operations are largely well established. Leveraging existing techniques to address practical challenges is reasonable, provided that the proposed designs are motivated by specific limitations of prior methods and their effectiveness is convincingly demonstrated. However, the manuscript lacks a clear motivation and fails to sufficiently distinguish the proposed approach from existing techniques. In essence, the framework primarily applies established self-supervised pretraining paradigms to cross-dataset invasive neural signal modeling, with tailored input organization and backbone architecture to accommodate data heterogeneity.

2.A major concern is that the experiments do not convincingly demonstrate cross-domain generalization. As stated in Section 3.2, all pretrained models undergo full-parameter fine-tuning using 80% of the target-domain data. Thus, the cross-species, cross-task, and cross-region evaluations essentially assess performance after source-domain pretraining and target-domain fine-tuning, rather than direct generalization to unseen domains. This distinction is fundamental. The current results demonstrate UniBCI's potential for cross-domain transfer through fine-tuning, but do not establish its ability to perform unified neural decoding without target-domain adaptation.

Questions for authors
  1. Could the authors further clarify the key motivation behind CST and IAA, particularly how their designs address limitations of existing methods and what distinguishes them from prior approaches?

  2. Could the authors provide additional evidence, such as frozen-backbone or zero-shot evaluations, to substantiate the claimed cross-domain generalization without target-domain adaptation?

  3. Could the authors conduct controlled experiments with matched pretraining data volumes to isolate the contribution of data diversity from that of data scale in Figure 4?

2 comments

(1) Motivation and novelty of CST and IAA.

We appreciate the reviewer's observation that the individual operations underlying CST and IAA are established techniques.

For CST, unlike conventional approaches that assign an independent learnable ID embedding to each subject or session (e.g., NDT2), UniBCI leverages a pretrained language model to encode natural-language recording metadata. This design is motivated by the semantic relationships among different recording conditions, including species, brain regions, and tasks. Rather than treating each recording condition as an isolated categorical ID, semantic metadata embeddings provide a shared representation space in which related recording conditions can be associated (shown in Appendix J). Furthermore, new recording conditions can be incorporated through their metadata without introducing and training an additional independent ID embedding from scratch (shown in Figure 6). Our ablation experiments demonstrate the effectiveness of metadata conditioning, and the additional comparison with learnable categorical metadata embeddings further isolates the contribution of language-based semantic encoding. The corresponding results are presented as

VariantM1-CO1 (Multi-day), BAM1-CO1 (Cross-day), BAMC-Maze, R²LICK, BAPPC-FINGER, BAArea2-Bump, R²
w/ ID embed0.869±0.0080.821±0.0090.865±0.0020.736±0.0060.957±0.0030.875±0.002
w/ Metadata0.900±0.0080.855±0.0140.886±0.0030.759±0.0030.968±0.0030.888±0.002

For IAA, its design is motivated by the spatiotemporal properties of invasive neural recordings: (i) neural spike activity exhibits fine-grained temporal dependencies within short intervals, and (ii) neural population activity exhibits structured local interactions across recording areas and temporal intervals. Our previous ablation results show that ILA and ASWA components in IAA contribute to downstream performance. In this revision, we add one ablation experiment of replacing IAA with conventional spatiotemporal full attention. The results show

VariantM1-CO1 (Multi-day), BAM1-CO1 (Cross-day), BAMC-Maze, R²LICK, BAPPC-FINGER, BAArea2-Bump, R²
w/ Tempo-spatial Full Attention0.611±0.0120.539±0.0110.720±0.0230.722±0.0080.862±0.0050.777±0.010
w/ IAA0.900±0.0080.855±0.0140.886±0.0030.759±0.0030.968±0.0030.888±0.002

Replacing the whole IAA module with traditional tempo-spatial full attention leads to severe performance degradation and high latency across all datasets, proving that global attention introduces unnecessary noise for neural dependencies.

We have clarified these motivations in the revised manuscript and presented details in Figure 5.

(2) Cross-domain Transfer and Target-domain Adaptation.

We thank the reviewer for raising this important point. We would like to clarify that the goal of UniBCI is cross-domain transfer learning, rather than zero-shot cross-domain generalization. Our objective is to investigate whether a neural representation pretrained on heterogeneous intracortical recordings can be transferred and adapted to new recording domains and decoding tasks.

We believe that this distinction is particularly important for intracortical BCI. Neural recordings can differ substantially across domains due to differences in species, subjects, brain regions, and experimental tasks. More importantly, these differences may extend beyond the input signal distribution: different tasks can involve different behavioral variables, decoding objectives, and even output dimensions. Consequently, a decoder trained on one domain may not be directly applicable to another domain without adaptation. In such settings, requiring zero-shot decoding is not always an appropriate or practically meaningful evaluation criterion.

Our evaluation protocol therefore follows a target-domain adaptation setting, in which the pretrained model is fine-tuned using data from the target domain. To further demonstrate data efficiency, we conducted few-shot experiments (20% training data) in Appendix I.2 (Tables 5 & 6). UniBCI consistently outperforms baselines across transfer tasks, proving our pretrained representations enable rapid adaptation with minimal target data.

(3) Effect of pretraining data diversity versus data scale.

We agree that our original Figure 4 does not fully isolate the contribution of data diversity from that of data volume, since expanding the pretraining corpus simultaneously increases both.

To address this issue, we conduct additional matched-volume experiments. Specifically, we construct pretraining subsets with multiple same total recording durations (7.5h, 12.5h, 17.5h) but different levels of diversity: (i)M1-CO1, (ii)M1-CO1 and M1-CO2, (iii)only Macaque, (iv) Macaque and Mouse. The results show that

PPC-FINGER(Balanced Accuracy)

Experiment7.5 hours12.5 hours17.5 hours
M1-CO10.8530.8900.916
M1-CO1 and M1-CO20.8880.9160.933
only Macaque0.9360.9440.947
Macaque and Mouse0.9440.9480.950
From scratch (reference)0.9210.9210.921

Visual-Coding(Balanced Accuracy)

Experiment7.5 hours12.5 hours17.5 hours
M1-CO10.4500.4420.654
M1-CO1 and M1-CO20.5260.6720.693
only Macaque0.7290.7350.739
Macaque and Mouse0.7350.7390.746
From scratch (reference)0.6690.6690.669

The results reveal two complementary observations. First, increasing pretraining data volume generally improves downstream performance when the corpus maintains sufficient diversity. Second, at matched data volumes, increasing the diversity of recording subjects, task paradigms, and species consistently improves transfer performance across both benchmarks.

In particular, adding a different subject performing the same task improves performance, while incorporating additional macaque datasets with diverse task paradigms yields further gains. Cross-species pretraining achieves the strongest results.

Notably, pretraining exclusively on M1-CO1 underperforms training from scratch, suggesting that a larger volume of narrowly distributed data does not necessarily produce transferable representations. These results provide additional evidence that the transferability of pretrained neural representations depends not only on data volume but also on the diversity of the pretraining corpus.

We have incorporated these results into the revised manuscript and moderate the corresponding Section3.3 and Figure 4.

Reviewer_dTyCOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes UniBCI a unified pretrained model on invasive neural spike data for Brain-Computer Interfaces. The UniBCI model combines neural spike data with recording metadata to create a context-conditioned spatio-temporal tokenization scheme. It uses an Interval-Area Attention mechanism that combines linear and sliding-window attention to capture temporal and spatial dependencies. The model is trained using a self-supervised masked reconstruction objective. UniBCI is pretrained on macaque and mouse recordings and demonstrates transfer across diverse new recordings across subjects, sessions, tasks, and human recordings.

Critical strengths
  • The paper addresses an important problem of learning transferable representations that generalize
  • Although limited, there are performance gains demonstrating diverse cross-generalization
  • The proposed model achieves competitive performance with relatively few trainable parameters
Critical weaknesses
  • The main contributions of this work are primarily engineering contributions that don't provide deeper insights about the data. Many of these engineering techniques have significant overlap with prior works, which makes it difficult to identify the specific novel contribution of this work's method.
  • The performance gains over previous methods are generally small and based on error margins in table results are often possibly not significant. Additional significance testing or contextualization of the magnitude of the performance increases would be helpful
  • The CST tokenization substantially overlaps the NDT2 method, which already incorporates session, subject, and task context embeddings.
  • The Interval-Area Attention does not have a strong justification or ablation experiments to explain its advantage over other attention methods.
  • Masked reconstruction is also an established method for spikes used in NDT1, NDT2, MtM, as well as for other recording modalities, such as in BrainBERT: Self-supervised representation learning for intracranial recordings Wang et al. 2023
Questions for authors

Which model contributions or combination of contributions are most novel and can inform future model development? Why is Interval-Area Attention the best choice? Could the authors contextualize the magnitude of the performance gains and indicate which gains are significant?

3 comments

(1) Novelty and contributions of UniBCI.

We agree that several individual components of UniBCI are built upon established techniques, including masked signals reconstruction.

Instead, the primary model contributions of UniBCI lie in tokenization combined with context of natural language and neuroscience-motivated integration into feature-learning for heterogeneous invasive neural recordings. The former one enables conditional representing across heterogeneous recording conditions within a semantically rich embedding space. The later one separates fine-grained temporal modeling within short intervals from local population-level interactions across intervals and recording brain areas.

UniBCI offers several design principles that may inform the development of future neural foundation models. First, semantic metadata conditioning provides an alternative to independent subject- or session-specific embeddings, enabling models to represent relationships among heterogeneous recording contexts and accommodate new conditions more flexibly. Second, the Interval-Area Attention mechanism illustrates the potential of structure-aware and hierarchical attention, in which modeling operations are tailored to the local temporal dependencies and spatial organization of neural population activity rather than relying exclusively on global attention. As broader design principles for future foundation models, semantic metadata conditioning and data-structure-aware architecture design are motivated.

We have revised the manuscript to clarify these contributions and avoid overstating the novelty of individual building blocks.

(2) Relationship between CST and NDT2.

Unlike traditional approaches that assign an independent, learnable ID embedding to each subject or session, such as NDT2, our approach leverages a pretrained language model to semantically encode natural-language metadata. This design offers several potential advantages. On the one hand, semantic relationships across species, brain regions, and tasks can be captured within the text embedding space (shown in Appendix J.1). Such sparse but semantically rich representations of natural language enrich the latent space of invasive signals (shown in Appendix J.2). On the other hand, new recording conditions can be incorporated without requiring additional independent ID embeddings trained from scratch. It is time-saving to decode neural signals and portal with less parameters for transfer-learning in the real application of invasive BCIs (shown in Figure 6).

In first submission, we ablated conditional metadata and proved that it helps mitigate domain shift. To better isolate the contribution of this design, we additionally conduct a controlled experiment with learnable categorical metadata embeddings. The results show that

VariantM1-CO1 (Multi-day), BAM1-CO1 (Cross-day), BAMC-Maze, R²LICK, BAPPC-FINGER, BAArea2-Bump, R²
w/ ID embed0.869±0.0080.821±0.0090.865±0.0020.736±0.0060.957±0.0030.875±0.002
w/ Metadata0.900±0.0080.855±0.0140.886±0.0030.759±0.0030.968±0.0030.888±0.002

Replacing natural-language metadata with learnable ID embeddings degrades performance across benchmarks, confirming that semantic embeddings offer richer priors than categorical vectors. More details of ablation study are presented in Section 3.4.

(3) Motivation and effectiveness of Interval-Area Attention.

We appreciate the reviewer’s concern and agree that the motivation and empirical justification of IAA should be more clearly presented.

Prior studies extract features of invasive neural recordings with tempo-spatial full attention or grouped attention. Those methods consider global relationship in both time and channel dimensions, but heavy computational complexity hinders application of invasive BCIs which requires lightweight deployment and efficient decoding. IAA is motivated by two complementary properties of invasive neural recordings: (i) Neural spike activity exhibits fine-grained temporal dependencies within short time intervals. (ii) Neural population activity exhibits structured local interactions across neighboring recording areas and temporal intervals. Our existing ablation results demonstrate that removing ILA leads to an 8.6% performance decrease on M1-CO1 (Multi-day), while removing ASWA results in consistent degradation across downstream benchmarks. Replacing ILA with full attention also increases inference latency and reduces performance.

To further validate this design, we conduct one additional ablation experiment of replacing the whole IAA module with traditional tempo-spatial full attention. The results indicate that

VariantM1-CO1 (Multi-day), BAM1-CO1 (Cross-day), BAMC-Maze, R²LICK, BAPPC-FINGER, BAArea2-Bump, R²
w/ Tempo-spatial Full Attention0.611±0.0120.539±0.0110.720±0.0230.722±0.0080.862±0.0050.777±0.010
w/ IAA0.900±0.0080.855±0.0140.886±0.0030.759±0.0030.968±0.0030.888±0.002

Replacing the whole IAA module with traditional tempo-spatial full attention leads to severe performance degradation and high latency across all datasets, proving that global attention introduces unnecessary noise for neural dependencies. This observation aligns with our window sensitivity analysis in Appendix I.4 (Figure 7), which demonstrates that neural dynamics are predominantly governed by local temporal dependencies and expanding the attention range fails to bring gains while introducing redundant computation and long-range noise.

(4) Significance and magnitude of performance improvements.

We appreciate this important observation. We agree that the performance improvements are not uniform across all downstream tasks, and that some improvements are relatively small. For each dataset and evaluation metric, we performed a two-sided paired t-test to compare UniBCI against each baseline method across five random seeds (df=4df = 4). Runs with the same random seed shared identical data splits and model initialization. We additionally report Cohen's dzd_z as the effect size, defined as the mean of the paired differences divided by their standard deviation. On PPC-FINGER, UniBCI achieves a 0.34 percentage-point improvement in balanced accuracy over the best-performing baseline, NDT2 (dz=1.39d_z = 1.39, paired tt-test, p=0.036p = 0.036). On Visual-Coding, UniBCI outperforms the best baseline, MtM, by 0.86 percentage points (dz=1.41d_z = 1.41, p=0.034p = 0.034), also achieving statistical significance. On MC-Maze, UniBCI improves R2R^2 by 0.22 percentage points over the best baseline, NDT1 (dz=1.27d_z = 1.27, p=0.047p = 0.047), with a statistically significant difference. Although the improvements over the best-performing baselines on M1-CO1 (Multi-day) and Perich T-CO do not reach statistical significance, UniBCI still significantly outperforms the other baseline methods on these datasets. Overall, UniBCI achieves better average performance across all downstream tasks than existing baseline methods.

Noticeable, as shown in Figure 6, UniBCI also reaches performance competitive with existing methods while using a very small number of trainable parameters and achieving low latency. Furthermore, our data-scaling experiments suggest that increasing the amount and diversity of pretraining data can lead to further performance improvements. The effectiveness of pretrain data volume and diversity is presented in Section 3.3.

I thank the authors for the detailed rebuttal and additional experiments. I appreciate the new ablations and statistical significance analysis which address some of my concerns.

However, my main concern remains the limited methodological novelty. The new experiments better support the effectiveness of semantic metadata embeddings and IAA, but I believe the approach still primarily combines established techniques. Also, the improvements over the strongest baselines remain modest, even where statistically significant.

Overall, I acknowledge the engineering contributions and improved experimental justification, but I do not find the rebuttal sufficient to change my review. I will maintain my score.

Comments on the paper

1 comment
Area_Chair_QrYZOpenReview ↗

Dear Reviewers, please engage in the discussion of this paper

Dear Reviewers, the Authors have repsonded to your questions. I would be inteterested to see if you find your questions addressed and if that changes your valuation of the paper.

Your AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.