R-NDT: Robust Neural Data Transformer for Generalized Causal Motor Decoding

Intracortical brain motor decoding aims to interpret and translate neural spike activity into behaviors. Causal decoding models that can generalize across variations, such as sessions, behaviors, subjects, and recording sites, are critical for real-world brain-computer interface (BCI) applications. However, every recording session presents the decoder with a new set of neural units and a continuous stream of nonstationary population spike activity. In this work, we present R-NDT, a robust neural data transformer that is designed to adapt to new recording sessions without need to re-learn unit/session embeddings, and it decodes long-term recordings in a streaming causal manner. Specifically, 1) unit identity is inferred from a few seconds of unlabeled calibration activity by a permutation-equivariant encoder to weight units in a population read-in; 2) a sliding-window causal transformer processes spike count bins with a bounded key-value cache; 3) a moving-average state conditions every block through adaptive layer norm; and 4) behavior covariates read the latent state through cross-attention. We pretrained R-NDT using several public datasets. After pretraining, we evaluated R-NDT generalization by fine-tuning it on cross-session, cross-type, cross-subject, and cross-site downstream behavior decoding tasks. Our results show that, on average across 20 sessions, R-NDT outperforms the decoding performance of existing decoding models, which has implications for future BCI decoders.