Pretrained Neuron-Level Representations Enable Across-Subject Zero-Shot Transfer of Neural Decoders

Pretraining neural decoders across electrophysiology recordings offers a path toward models that generalize to unseen subjects, yet existing approaches typically require adaptation to each new neural population. A key obstacle is that spike-sorted recordings lack a shared input space. Each recording contains a new, unordered population of neurons, which existing decoders typically represent using recording-specific parameters. We introduce Zen, a framework that replaces these parameters with transferable neuron representations inferred by a pretrained neuron-level encoder. A decoder trained over this space then operates directly on the new population, without target-specific gradient updates, alignment, or behavioral labels. We evaluate three neuron-level encoders and three decoder architectures, including our proposed Time-Patch decoder. Time-Patch preserves neuron-specific tokens throughout the Transformer backbone, whereas prior architectures aggregate activity across neurons before the Transformer backbone. Our strongest configuration, Zenith, combines NuCLR representations with Time-Patch and achieves state-of-the-art cross-subject zero-shot decoding on the IBL Brain-Wide Map and Allen Visual Behavior Neuropixels datasets, reaching approximately 83% and 85% balanced accuracy on reward decoding, respectively. Under the finetuning setting, Zenith surpasses the best fully finetuned baselines on IBL wheel speed and reward decoding with only 25% of the available target finetuning trials. These results establish transferable neuron representations as a practical basis for both zero-shot neural decoding and data-efficient supervised adaptation.