- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Authors propose a new model architecture that uses trial structure to define a set of cross-attention pooling tokens which aggregate information across neuronal population activity across structured time bins of the trial.
Critical strengths
- The problem formulation is an important direction in neuroscience as prior models usually rely on labeled data collection and fine-tuning to achieve behavior decoding.
- Authors perform extensive ablation on the different modeling components and highlight which design choice matter more than others.
Critical weaknesses
- I have significant concerns about the write-up. I think the write-up requires a significant rewrite and it doesn't meet the high bar of ICLR right now. I found many sentences very hard to read and structurally ill-posed.
- The manuscript fails to describe the details of the proposed methodology in the main text. The model formulation is just limited to one algorithm table, where the definitions of the learnable parameters are dumped into the Appendix. I understand that they are limited to 9 pages, but I think they should still be able to describe their methodology within the limits.
- Even though I am familiar with the domain and can understand what the authors try to convey in the related work, I dont think it does a good job on explaining the prior literature, especially for someone who is not very familiar with this domain. they should explain the relation of each work to the proposed problem more clearly.
- Can they explain what they mean by "a unit with two properties: (1) a single decoder can use it in every session, and (2) it carries behavioral meaning. We look for such a unit in population activity, the joint activity of the recorded neurons". Similar to SPINT, or POYO, their model also relies on cross-attention across neuronal population. In their model formulation, are they using multi-unit spikes rather than sorted units? if so, I'd like to see the proposed model's performance with sorted spikes where the neuron identity changes from one session to another. I wonder if the cross-attention module will be able to transfer its knowledge to a new session without fine-tuning.
- In line with my previous comment, what modeling component here makes their cross-attention module work but other models' fail?
- In terms of methodology, my primary concern is that in Table 2, it looks like the model heavily relies on the state boundaries. These are unknown in causal and real-time settings, and using this information for inference significantly hinders models applicability for real-world neuroscience settings.
- In L154-155, they say: "This step uses only the unlabeled activity of that session", but aren't they using b_stim and b_move, which are supervised variables requiring task knowledge? Also, in the abstract, they also say "A tokenizer for new sessions needs a unit that every session shares and that carries behavior", which is not necessarily accurate when fine-tuning is allowed.
- The baseline results are also concerning, I cannot see any setting where POYO and CEBRA achieves reasonable performance except for table 17 where the authors provide the event boundaries. This makes me think that if something in their preprocessing and/or normalization is the reason why the baseline models fail.
Questions for authors
Please see weaknesses above.