- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an extension to a recent method, sparse component analysis, such that it can be used on single-trial spike count data without averaging or pre-smoothing. It adapts SCA primarily with a Poisson reconstruction loss (with sparsity and orthogonality regularization). It is compared against standard methods such as GPFA and LFADS, showing that pSCA recovers the groundtruth synthetic latents accurately with much less training time, and behaviorally isolated latents in monkey hand-cycling and mouse free behavior datasets.
Critical strengths
- overall a very well-written and clearly structured paper. I think the LLM-assisted polishing introduced some oddities (like the repeated usage of "researchers" in the intro), but as a whole unproblematic. Similarly, the figures are of high quality.
- the paper proposes a fairly straightforward and obvious extension to SCA, and shows that it indeed produces spatiotemporally isolated latents while incurring significantly shorter runtime than LFADS and GPFA
- it is evaluated on two real datasets, one standard and one relatively novel, and finds latents that correlate well with intuitive aspects of behavior or EMG
Critical weaknesses
-
the proposed extension is relatively incremental over SCA, which is not a problem by itself, and the simplicity and intuitiveness of the proposed method are advantages. But given the limited methodological innovations, the evaluation is then fairly limited in my opinion.
-
Essentially, pSCA finds the correct GT latents in a synthetic task, and produces more behaviorally-isolated latents in the monkey and mouse datasets. It is acknowledged that the monkey dataset is a well-used benchmark with clear expectations for what one should find, nevertheless, showing that latents have selectivity for prep vs. move or direction of cycling doesn't say much about the method. Though it is interesting that it's much better than the other methods, including SCA.
-
Similarly, for the mouse dataset, (cortical) latents are found to be isolated to labeled behavioral motifs, but this is only done in a qualitative fashion.
-
more generally, I think comparison to more recent baseline methods for unsupervised latent variable models in neuroscience may be warranted, including those that do require pre-smoothing, many of which the authors cite. LFADS and GPFA are both now more than 10 years old.
-
I would have liked to see more interpretation of the latents beyond "they correlated to individual behavioral dimensions". For instance, where are the neurons that participate strongly in each latent, what is the spread of loading for each neuron in the first place, is that biologically plausible, what do the learned conv filters look like for the real datasets, how do the results change with latent dimensionality, etc. The methodological innovation and evaluation are a little thin for an ICLR paper, so I would then at least expect to see some level of scientific discovery.
Questions for authors
Technical clarifications:
- If I understand correctly, the timepoint-wise MLP’s takes as input the spike trains, and its output is then convolved with the learned filter. Are the conv filters of the encoding model causal? Also, what is meant by “the filter weights are learned independently at each window position”? As in their values can change relative to absolute time, or simply that every value in the kernel is free to change flexibly as in a standard learned convolution in deep learning, but subject to l1 normalization?
- lambda_sparse is adaptive but lambda_orth requires hyperparam tuning? But if I understand correctly, the sparsity term is adaptive in the sense that it’s pinned to the reconstruction loss by a fixed proportionality, which is itself a hyperparameter?
- how to choose latent dimensionality k?
More broadly, see the last two points in weaknesses, i.e., 1) further and more competitive baselines published in the last 5 years, and 2) scientific interpretation of learned latents / filters / loadings beyond "they correlate well with behavior".
10 comments
Re: Comparisons to more recent baselines.
“More generally, I think comparison to more recent baseline methods for unsupervised latent variable models in neuroscience may be warranted, including those that do require pre-smoothing, many of which the authors cite. LFADS and GPFA are both now more than 10 years old.”
We thank the reviewer for this suggestion, and believe this addition substantively strengthens the manuscript. To include a recent, widely used latent variable model, we fit CEBRA (Schneider et al., 2023) to the cycling dataset. CEBRA uses contrastive learning to pull together neural activity patterns associated with similar behavioral variables or nearby time points, and to push apart those associated with dissimilar behaviors or distant time points. We fit both a fully unsupervised variant (CEBRA-time) – which would be most comparable to pSCA and the other unsupervised comparisons – and also a variant supervised by EMG and kinematics (CEBRA-behavior) (Figs. 2 and A.15).
Of the competing methods we tested, other than SCA, CEBRA is the only one explicitly designed to produce interpretable latent representations. Even so, it did not demix the processes in this dataset. CEBRA-behavior captures externally observable aspects of the task, such as cycling, but fails to isolate internal processes such as movement preparation, and preparation to stop, which pSCA recovers without behavioral supervision (Fig. 2). CEBRA-time demixes poorly as well (Fig. A.15), and both variants predict held-out neurons worse than methods with a reconstruction term in their objective function (pSCA, GPFA, and LFADS). Across all methods, pSCA remains the most effective at balancing predictive performance, training speed, and demixing (Fig. 2). In response to another reviewer, we also applied post-hoc sparsifying transformations (varimax and SCA) to the latents learned by GPFA, LFADS, CEBRA-time, and CEBRA-behavior. Even after these transformations, pSCA's latents remained the most demixed (Figs. A.16–A.18).
We believe all of the CEBRA results highlight the strengths of our approach. We currently show CEBRA-behavior in Fig. 2 and CEBRA-time in Appendix, but have no strong preference about which variant appears in the main text. If you find a particular one more compelling, we are happy to feature it instead.
Re: Scientific interpretation of the learned latents.
“I would have liked to see more interpretation of the latents beyond "they correlated to individual behavioral dimensions". For instance, where are the neurons that participate strongly in each latent, what is the spread of loading for each neuron in the first place, is that biologically plausible, what do the learned conv filters look like for the real datasets, how do the results change with latent dimensionality, etc. The methodological innovation and evaluation are a little thin for an ICLR paper, so I would then at least expect to see some level of scientific discovery.”
We thank the reviewer for pushing us on this point. To move beyond correlating latents with behavior, we fit pSCA separately to fM1 and DLS recordings from four additional mice. We elaborate on these results in the main text (new Fig. 4; Section 3.4) and appendix (Figs. A.26, A.28–A.31), and briefly summarize the key findings here:
-
) Consistent latents across animals and regions. In four of the five mice, pSCA found an eating-selective latent in both fM1 and DLS that closely tracked muscle activity (Fig. 4A). Because each region of each animal was fit separately, and the striatal recording location differed from mouse to mouse, this recurrence indicates that pSCA recovers structure that generalizes across brain regions and animals.
-
) Reproducible inter-regional timing. Identifying the same latent in each region allows its timing to be compared directly. During eating, fM1 leads DLS by an identical 80 ms in three of the five mice (Fig. 4A), whereas at rearing offset a striatal transition latent peaks before the cortical one, also generalizing to held-out bouts (Fig. 4D) - see Fig. A.31 for another striatal transition signal within a different mouse.
-
) Biologically plausible loadings. In response to the reviewer’s question about where contributing neurons are located, we quantified the spatial clustering of each eating latent’s loadings along the probe with Moran’s I (Fig. 4B,C; Section A.6.6). In every mouse, striatal eating latents were carried by spatially clustered neurons, whereas cortical ones were distributed across the population, consistent with the convergent organization of corticostriatal projections.
-
) A testable hypothesis. Together, these results suggest that cortex leads striatum within behaviors requiring moment-to-moment forelimb control, whereas striatum leads cortex at transitions between behaviors, consistent with the proposed role of striatum in action selection (Markowitz et al., 2018). This hypothesis rests on pSCA identifying the same process independently in each region and animal, something methods with latents co-active across all behaviors, such as GPFA (Fig. 3B), cannot provide.
Thanks for the feedback and acknowledging that simplicity and intuitiveness can be advantageous. Indeed, our goal of the method is to have something that is easily usable and intuitive for the neuro community. Our paper thus focused on including the four essential model innovations expanding on the original SCA framework (Section 2) that enable the model to work effectively out-of-the-box on several datasets, rather than extra bells-and-whistles.
In terms of the limited evaluation, based on your feedback (and from other reviewers), we have now substantially increased the evaluation of the method in our revision. This includes providing comparisons to more recent methods (CEBRA) and substantial further scientific exploration of the mouse naturalistic arena dataset (including on 4 additional mice), which is detailed in further responses below.