- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
I am not a BCI expert
The goal is to great a multi-modal multi-participant cortical decoder trained on handwriting, speech, cursor, and typing. It can handle variable channel counts. The novelty comes from the training across modalities as the components are all previously descirbed. It also aggregates pretty much all available Utah array data.
Critical strengths
The multimodal training is a strength and novel The large (for the available data) data set is a strength The engineering is nice to bring it all together
Critical weaknesses
The SOTA claims are week given the small number of seeds and the small number of validation trials. Boot strappingn is needed to make strong claims.
The cursor targets are the original decoded targets, not the indented. This seems circular and of unclear value.
I don't understand the claims of "universal" as each new individual would require embeddings and fine-tuning. We don't see few shot data.
Questions for authors
Please use more seeds and better estimates of confidence bounds. Increase the size of the vlidation set. Please make clear what is needed for each new participant and lessen claims of "universailty"
2 comments
Gentle Reminder
Dear Reviewer,
As the discussion period is approaching its deadline, we would appreciate any feedback on our rebuttal, particularly whether it addresses your concerns. We would be happy to clarify any remaining points.
Thank you, Authors
We thank the reviewer for the review and for the clear list of concerns, each of which led to an addition.
Our comparison follows the standard practice on these datasets: we train and evaluate on the same splits as the published studies and compare the resulting numbers one to one. On that basis CorticAll is numerically better on 7 of 8 datasets. Since the validation sets are fixed by the original releases and cannot be enlarged, we added bootstrap confidence intervals as a proxy for more validation trials: they show how much each dataset result could move with a different sample of trials (10,000 resamples of the validation trials; error in %, 95% interval):
For five of the seven datasets (counting the Kunz and the Jude typing releases once each) the advantage is robust to resampling of the validation trials; for Card and Willett speech the published value lies within our interval. Intervals for every result are in App. A.7. Additional seeds were not feasible within the rebuttal period, since each seed requires retraining every regime on every dataset (App. A.6).
As stated in Section 2.1, the cursor targets are the trajectories decoded in the source studies. These datasets were recorded during online cursor control: the participants steered a decoded cursor to on-screen targets, and the cursor's trajectory is the only movement signal that was recorded. The revision adds why we include cursor: to test whether a continuous control modality can share weights with speech, handwriting and typing. It can, with R² up to 0.93.
CorticAll is universal in its inputs and outputs: it accepts any configuration of intracortical arrays and channel sets, and decodes the four major communication BCI modalities with one set of weights. This is what allows a single model to be trained jointly across participants and modalities, the purpose it was built for; the definition is now given where the term is first used. Adding a participant who was not part of training requires labelled data from them: new channel embeddings are learned and the rest of the model is fine-tuned. New experiments with participants held out of pretraining show what this takes (Table 2, App. A.10). PER (%) at full target data, by which parameters are trained:
New channel embeddings alone are not enough; the spatial front-end has to adapt as well. Decoding rises above chance only from about 30 to 60 minutes of labelled data (Figure 4). Beyond that, pretraining on other participants reduces the data needed for some participants and not for others (Figure 4, App. A.10).