- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces two new innovations on the SLDS model family: a multi-lag moment intialization, and a regime-conditioned Laplace EM strategy for inference. The benefits of these innovations are first demonstrated on synthetic data, where both are shown to improve recovery of timescales from a dynamical system.
Critical strengths
SLDS methods can be highly initialization dependent, and all inference methods are approximate. These two facts mean that this powerful family of models can be difficult to use in practice--not just fitting them, but trusting the fits enough to extract meaningful insights into the behavior of the system. MTS-SLDS takes both of these issues seriously and provides an intuitive approach for addressing each.
Critical weaknesses
It only became apparent to me partway through the methods that MTS-SLDS isn't actually a new model structure; it's just a vanilla SLDS with a new learning/inference procedure. Even though this is technically described correctly throughout the text, it was not obvious to me early on that this is the case.
The text in the figures, specifically the axis labels, are impossible for me to read
The motivation for this technique is that it does a better job at timescale recovery than the standard SLDS fitting procedure. This is clearly demonstrated with the synthetic data. However, on the neural datasets, in both cases the authors find the timescales extracted by MTS-SLDS and SLDS to be comparable, and say so explicitly (L393, L419). Do the differences in timescale predictions actually lead to meaningfully different scientific conclusions in these cases?
Questions for authors
L66: "with limited data the recovered dynamics are not uniquely determined" - is there a ref for this?
In Fig 2B,C, why is timescale recovery similar between SLDS/MTS-SLDS for Gaussian observations but not for Poisson? Is there a mathematical explanation for this?
Synthetic data results use K=2 in the switching systems, but when I see these types of models applied to real data the number of states of often much higher (K=5-10). How does MTS-SLDS recover timescales as the number of states increases? How much more data is needed to accurately recover timescales?
Another comment on synthetic data: the timescales used to generate data are short relative to the time window (4 sec). I can imagine many scenarios where timescales are longer and time windows are shorter in real data. How do model fits and timescale recovery degrade as the time window is shortened? Can this lead to guidance on when the model will or won't work on real data?
The synthetic data experiments should include some ablations; for example, what happens if you initialize with multi-lag moment but then fit the standard way without RC-L-EM? What about the opposite? Are improvemets coming from one of these steps or both? [EDIT: I see this is addressed in Appendix F, this should be mentioned in Sec. 3]
How does MTS-SLDS and SLDS compare under model misspecification? For example, what if you generate synthetic data with more states and timescales, and fit SLDS models with fewer states? What about the opposite scenario? Is MTS-SLDS more robust in these regimes?
Why didn't the authors fit SLDSs with K>1 to the V4 data?
For SS data, why was K=2 chosen?
Recurrent SLDS models are a powerful alternative to standard SLDS models. Can the ideas presented here be transferred to recurrent models? How does an rSLDS perform on the real data - does it have higher likelihoods? Find similar timescales?
5 comments
Q1 response: With finite and noisy observations, latent dynamical parameters and their spectra can be poorly constrained, so different estimated dynamics may provide similar predictive likelihoods. This is a well-known challenge in system identification. We have added two relevant references[1,2] in the manuscript.
- Tsiamis, A., & Pappas, G. J. (2019). Finite Sample Analysis of Stochastic System Identification.
- Tsiamis, A., & Pappas, G. J. (2021). Linear Systems Can Be Hard to Learn.
Q2 response: We think the difference may mainly come from the difficulty of inference under Poisson observations. For Gaussian observations, the measurement model is linear with Gaussian noise, which allows exact Gaussian state inference conditional on a fixed regime sequence. In contrast, Poisson observations have nonlinear emission rates and state-dependent noise, which leads to non-Gaussian posteriors, especially under sparse spikes. This can make conventional SLDS variational inference very sensitive to posterior approximations and can affect the recovered dynamical spectra. MTS-SLDS could mitigate these difficulties through multi-lag moment initialization and regime-conditioned inference, which presents great empirical improvements under Poisson observations.
Q 3, 4 response: We thank the reviewer for these important questions. Both questions concern the data requirements for reliable timescale recovery. In general, recovering regime-specific dynamics requires sufficient observations within each regime, particularly relative to the slowest timescale of interest. We have conducted additional simulations to investigate both questions.
- Increasing the number of regimes (Fig. 17). We extended the synthetic experiments to (K=1–5), adjusting trial lengths to maintain approximately 40 s of data per regime with all trials. MTS-SLDS achieved timescale errors of 3.2%, 6.4%, and 8.5% for (K=1,2,3), respectively. Performance gradually decreased for (K=4) (12.3%) and (K=5) (14.8%), but remained better than conventional SLDS.
- Decreasing the observation window (Fig. 16). We also tested trial lengths from 6s down to 1 s, with the slowest timescale fixed at 200 ms. With trial lengths decreasing from 6 to 1 s, MTS-SLDS timescale error increased from 7.8% to 15.1%, while regime accuracy decreased from approximately 0.97 to 0.83. Nevertheless, it maintained better timescale and regime recovery than conventional SLDS throughout the tested range. These experiments provide practical guidance on data requirements. Under the tested settings, reliable recovery generally required observation windows sufficiently longer than the slowest timescale, adequate data within each regime, and distinguishable regime dynamics. We have included these additional results in the Appendix F.6 (Figs. 16 and 17) and discussed the implications in the revised Discussion (lines 518–529).
Q5 response: We did have ablation experiments and had mentioned them briefly in Section 2.3 in the original submissions. We have now run more controlled comparisons to investigate the contributions of these two steps. We now included these results in Appendix F.1, Fig. 8 and also explicitly referred to them in Section 3, lines 345–349 as requested.
Q6 response: This is an interesting point. We conducted additional simulations to evaluate robustness to misspecification of both latent dimension and number of regimes. First, we generated data with and fitted models with – (Fig. 14). MTS-SLDS recovered the true timescale bands more consistently than SLDS, achieving lower errors across most tested dimensions. Even when was overestimated, the main timescale structure remained accurate, with errors of approximately 10% and 14% at and . Although additional spurious modes appeared, most were shorter than one sampling bin (5 ms). These sub-bin modes can be identified and excluded from timescale interpretation without removing the true modes, whose shortest timescale is 10 ms. In contrast, SLDS also produced incorrect timescale bands within the resolved range, which cannot be identified by this cutoff. We report the error in recovered true timescale bands and the number of additional spurious modes separately. Second, we generated data with and fitted models with – (Fig. 15). Underestimating merged distinct dynamics into a single spectrum. When , MTS-SLDS either left the additional regime nearly unused or split an existing regime while preserving its characteristic timescales. Timescale errors remained similar at and (8.1% and 8.2%), whereas SLDS showed substantially larger errors. These results suggest that MTS-SLDS is more robust to moderate model overparameterization, preserving the true timescale structure even when additional latent modes or regimes are introduced. We have included these experiments in the Appendix F.5 (Figs. 14 and 15).
Q7 response: We focused on sustained fixation in V4, where the behavioral condition was relatively stationary. Previous studies also treat this dataset as one stationary process, so we adopt this setting and characterize multiple timescales within a single dynamical regime.
Q8 response: We had evaluated different numbers of regimes and latent dimensions using held-out predictive performance. The (K=2) model improved prediction over (K=1), while (K=3) did not provide consistent additional improvements. We therefore selected (K=2), which also allowed us to characterize dynamical changes associated with movement onset and relate the inferred timescales to behavior. We thank the reviewer for pointing out that these model-selection details were not clearly reported in the original submission. We have now included the evaluation procedure and results in the Appendix H, Fig. 20.
Q9 response: We thank the reviewer for this suggestion. The main difference between SLDS and rSLDS is that rSLDS allows state-dependent regime transitions. The multi-lag initialization and regime-conditioned inference ideas of MTS-SLDS could be extended to rSLDS. As requested, we also fitted Poisson rSLDS to the Area 2 data using the same data and model dimensions K=2, H=6. rSLDS achieved comparable held-out predictive likelihood to MTS-SLDS and SLDS during active reaching and slightly higher likelihood during passive perturbations. It also identified similar onset-associated regime transitions and broadly comparable timescale ranges. However, among these three methods, MTS-SLDS showed a clearer active-passive contrast in movement-regime timescales and closer correspondence with the observed behavioral decay timescales. These results suggest that MTS-SLDS provides a more behaviorally informative characterization of neural timescales. We have included these comparisons in Fig. 22, Appendix H.
We thank the reviewer for these helpful comments.
Response W1,2: In this work, MTS-SLDS refers to the entire inference framework that builds on the standard SLDS generative model and integrates multi-lag moment initialization with regime-conditioned Laplace EM (RC-L-EM) to achieve more accurate and reliable recovery of neural timescales and switching regimes. We have revised the Introduction and relevant sections to make this distinction clearer and better emphasize the methodological contributions of MTS-SLDS. We have also enlarged the figures and increased the font sizes to improve readability.
W3 response: Although both methods identify broadly comparable timescale ranges, their estimates differ in ways that they can provide interpretation for neural dynamics. In V4, both methods identify three timescale, but with different estimated values. MTS-SLDS also reproduces the observed autocorrelation structure more closely, which suggests that its inferred dynamics better capture the temporal structure of the recordings (Fig. 3). In Area 2, the differences have clearer scientific implications. MTS-SLDS identifies distinct movement-regime timescales between active reaching and passive perturbation (188 vs. 99 ms), whereas SLDS estimates are much more similar (117 vs. 108 ms) across these behaviors. Moreover, the MTS-SLDS estimates closely match the corresponding behavioral decay timescales (168 vs. 86 ms), while SLDS and sPCF do not show the same agreement (Fig. 4). These findings suggest that MTS-SLDS provides more informative timescale estimates for characterizing condition-dependent neural dynamics and their relationship to behavior. All these findings have been stated in previous submissions, and we have further clarified these interpretations in the revised manuscript.