- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes R-NDT for causal motor decoding across heterogeneous intracortical recording sessions. It derives unit representations from a short unlabeled calibration segment using a permutation-equivariant set encoder, then maps a variable-sized neural population into one token per time bin. A sliding-window causal Transformer with a bounded KV cache and multi-timescale drift memory processes the resulting stream, while a query-based readout predicts behavior. The model is pretrained on 156 public sessions through masked spike-count reconstruction and subsequently fine-tuned with labeled data from each target session. Evaluation covers 20 held-out public sessions under a per-trial causal protocol, as well as T-RT and NPCS recordings under a continuous whole-block protocol. The full model achieves strong aggregate performance against the included baselines, and its smaller from-scratch variant is also competitive.
Critical strengths
Strength 1
A promising architecture evaluated under a practically relevant streaming protocol. The combination of a flexible population read-in, bounded-memory causal attention, and a slower drift state addresses important constraints of continuous neural decoding. The whole-block evaluation avoids resetting the decoder at every short trial and is therefore particularly relevant to deployment. Across several datasets, R-NDT performs strongly against recent baselines. Notably, the smaller R-NDT-SS model remains competitive without pretraining, which supports the value of the architectural design itself.
Strength 2
A useful route for learning from unlabeled neural recordings. The masked-count objective allows R-NDT to pretrain across many sessions without aligned behavioral labels. The paper reports improved downstream performance as the amount of pretraining data increases. This is a promising direction for incorporating larger collections of neural recordings, potentially including recordings from other tasks for which suitable behavioral labels are unavailable. Such broader cross-task scaling remains a future possibility rather than a result established by this paper.
Critical weaknesses
Critical weaknesses
1. The case for unit-level inputs is insufficiently established. R-NDT relies on sorted units and their calibration statistics, but does not compare this choice with channel-level or latent-alignment approaches in performance and preprocessing cost. CRRL, RAT, and SABLE offer relevant alternative approaches to cross-session representation learning. Channel signals can also drift; my concern is that the specific benefit of the unit-level design remains unclear.
2. The contributions of key components are insufficiently isolated. The paper lacks controlled ablations for the -based unit encoder and drift memory, making it difficult to determine which components drive the gains. In addition, R-NDT-SS and R-NDT (FT) differ in both model size and pretraining, so their comparison does not isolate the effect of pretraining.
Questions for authors
Questions for authors
- Why use sorted units as inputs? What measurable benefit does unit-level calibration provide over channel-level or latent approaches? How do spike-sorting cost and errors affect the results?
- Could the authors provide size-matched controls and component ablations? In particular, a randomly initialized model of the same size, a learned embedding lookup in place of -based embeddings, and a model without drift memory would materially affect my assessment.
- Which parameters are updated during supervised fine-tuning on a target session? If the set encoder and also receive gradients from behavior labels, can the authors disentangle the benefit of unlabeled calibration from subsequent supervised updates?
- What is the practical value of pretraining? In Table 2, R-NDT (FT) exceeds R-NDT-SS by approximately 0.013 in mean , with a much smaller difference on some datasets. Since these models also differ in size, what gain is attributable to pretraining at matched capacity? How does that gain compare with the pretraining compute cost?
I would be willing to raise my score if the authors provide controlled evidence clarifying the independent value of unit-level calibration and the sources of the performance gains.