acceptodds
Under review as a conference paper at ICLR 2027

Inferring Multi-Timescale Neural Dynamics with Switching Linear Dynamical Systems

Abstract

Neural activity often exhibits multiple timescales that can vary with behavioral states and task conditions. Identifying these timescales from neural recordings is important for better understanding neural computation and function. However, traditional approaches based on autocorrelation fitting are difficult to scale to high-dimensional population recordings and can become unreliable when neural dynamics change with behavior. State-space models have been a powerful framework for modeling high-dimensional neural population activity through latent dynamical systems, but standard formulations and inference methods do not explicitly account for multiple timescales and therefore do not guarantee accurate recovery of the underlying temporal structure. Motivated by these questions, we introduce the Multi-Timescale Switching Linear Dynamical System (MTS-SLDS), a framework for identifying regime-specific latent timescales from continuous or spiking neural observations. MTS-SLDS combines a multi-lag moment initialization, which captures temporal structure across multiple observation lags, with regime-conditioned Laplace-EM inference, which reduces mixing of dynamical statistics across uncertain regimes. Characteristic timescales can then be extracted directly from the eigenvalues of the learned latent transition matrices. In synthetic experiments with Gaussian and Poisson observations, MTS-SLDS accurately recovers timescales and switching structure over a range of signal levels. We further apply this framework on neural spike recordings from two behavioral tasks. During a fixation task with neural activity recorded from V4, the inferred timescales agree with estimates from a dedicated Bayesian autocorrelation method. In Area 2 motor cortex recordings during a reaching task, we find that MTS-SLDS identifies behaviorally aligned dynamical regimes and provides more interpretable regime-specific timescale estimates. These results establish MTS-SLDS as a reliable framework for investigating how neural timescales behave within and across behavioral states, and for studying their role in neural population dynamics.

Primary area:
applications to neuroscience & cognitive science
Keywords:
switching linear dynamical systems, multiple timescales, neural population dynamics, computational neuroscience

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YGdwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces two new innovations on the SLDS model family: a multi-lag moment intialization, and a regime-conditioned Laplace EM strategy for inference. The benefits of these innovations are first demonstrated on synthetic data, where both are shown to improve recovery of timescales from a dynamical system.

Critical strengths

SLDS methods can be highly initialization dependent, and all inference methods are approximate. These two facts mean that this powerful family of models can be difficult to use in practice--not just fitting them, but trusting the fits enough to extract meaningful insights into the behavior of the system. MTS-SLDS takes both of these issues seriously and provides an intuitive approach for addressing each.

Critical weaknesses

It only became apparent to me partway through the methods that MTS-SLDS isn't actually a new model structure; it's just a vanilla SLDS with a new learning/inference procedure. Even though this is technically described correctly throughout the text, it was not obvious to me early on that this is the case.

The text in the figures, specifically the axis labels, are impossible for me to read

The motivation for this technique is that it does a better job at timescale recovery than the standard SLDS fitting procedure. This is clearly demonstrated with the synthetic data. However, on the neural datasets, in both cases the authors find the timescales extracted by MTS-SLDS and SLDS to be comparable, and say so explicitly (L393, L419). Do the differences in timescale predictions actually lead to meaningfully different scientific conclusions in these cases?

Questions for authors

L66: "with limited data the recovered dynamics are not uniquely determined" - is there a ref for this?

In Fig 2B,C, why is timescale recovery similar between SLDS/MTS-SLDS for Gaussian observations but not for Poisson? Is there a mathematical explanation for this?

Synthetic data results use K=2 in the switching systems, but when I see these types of models applied to real data the number of states of often much higher (K=5-10). How does MTS-SLDS recover timescales as the number of states increases? How much more data is needed to accurately recover timescales?

Another comment on synthetic data: the timescales used to generate data are short relative to the time window (4 sec). I can imagine many scenarios where timescales are longer and time windows are shorter in real data. How do model fits and timescale recovery degrade as the time window is shortened? Can this lead to guidance on when the model will or won't work on real data?

The synthetic data experiments should include some ablations; for example, what happens if you initialize with multi-lag moment but then fit the standard way without RC-L-EM? What about the opposite? Are improvemets coming from one of these steps or both? [EDIT: I see this is addressed in Appendix F, this should be mentioned in Sec. 3]

How does MTS-SLDS and SLDS compare under model misspecification? For example, what if you generate synthetic data with more states and timescales, and fit SLDS models with fewer states? What about the opposite scenario? Is MTS-SLDS more robust in these regimes?

Why didn't the authors fit SLDSs with K>1 to the V4 data?

For SS data, why was K=2 chosen?

Recurrent SLDS models are a powerful alternative to standard SLDS models. Can the ideas presented here be transferred to recurrent models? How does an rSLDS perform on the real data - does it have higher likelihoods? Find similar timescales?

5 comments
AuthorsOpenReview ↗

We thank the reviewer for these helpful comments.

Response W1,2: In this work, MTS-SLDS refers to the entire inference framework that builds on the standard SLDS generative model and integrates multi-lag moment initialization with regime-conditioned Laplace EM (RC-L-EM) to achieve more accurate and reliable recovery of neural timescales and switching regimes. We have revised the Introduction and relevant sections to make this distinction clearer and better emphasize the methodological contributions of MTS-SLDS. We have also enlarged the figures and increased the font sizes to improve readability.

W3 response: Although both methods identify broadly comparable timescale ranges, their estimates differ in ways that they can provide interpretation for neural dynamics. In V4, both methods identify three timescale, but with different estimated values. MTS-SLDS also reproduces the observed autocorrelation structure more closely, which suggests that its inferred dynamics better capture the temporal structure of the recordings (Fig. 3). In Area 2, the differences have clearer scientific implications. MTS-SLDS identifies distinct movement-regime timescales between active reaching and passive perturbation (188 vs. 99 ms), whereas SLDS estimates are much more similar (117 vs. 108 ms) across these behaviors. Moreover, the MTS-SLDS estimates closely match the corresponding behavioral decay timescales (168 vs. 86 ms), while SLDS and sPCF do not show the same agreement (Fig. 4). These findings suggest that MTS-SLDS provides more informative timescale estimates for characterizing condition-dependent neural dynamics and their relationship to behavior. All these findings have been stated in previous submissions, and we have further clarified these interpretations in the revised manuscript.

Q1 response: With finite and noisy observations, latent dynamical parameters and their spectra can be poorly constrained, so different estimated dynamics may provide similar predictive likelihoods. This is a well-known challenge in system identification. We have added two relevant references[1,2] in the manuscript.

  1. Tsiamis, A., & Pappas, G. J. (2019). Finite Sample Analysis of Stochastic System Identification.
  2. Tsiamis, A., & Pappas, G. J. (2021). Linear Systems Can Be Hard to Learn.

Q2 response: We think the difference may mainly come from the difficulty of inference under Poisson observations. For Gaussian observations, the measurement model is linear with Gaussian noise, which allows exact Gaussian state inference conditional on a fixed regime sequence. In contrast, Poisson observations have nonlinear emission rates and state-dependent noise, which leads to non-Gaussian posteriors, especially under sparse spikes. This can make conventional SLDS variational inference very sensitive to posterior approximations and can affect the recovered dynamical spectra. MTS-SLDS could mitigate these difficulties through multi-lag moment initialization and regime-conditioned inference, which presents great empirical improvements under Poisson observations.

Q 3, 4 response: We thank the reviewer for these important questions. Both questions concern the data requirements for reliable timescale recovery. In general, recovering regime-specific dynamics requires sufficient observations within each regime, particularly relative to the slowest timescale of interest. We have conducted additional simulations to investigate both questions.

  1. Increasing the number of regimes (Fig. 17). We extended the synthetic experiments to (K=1–5), adjusting trial lengths to maintain approximately 40 s of data per regime with all trials. MTS-SLDS achieved timescale errors of 3.2%, 6.4%, and 8.5% for (K=1,2,3), respectively. Performance gradually decreased for (K=4) (12.3%) and (K=5) (14.8%), but remained better than conventional SLDS.
  2. Decreasing the observation window (Fig. 16). We also tested trial lengths from 6s down to 1 s, with the slowest timescale fixed at 200 ms. With trial lengths decreasing from 6 to 1 s, MTS-SLDS timescale error increased from 7.8% to 15.1%, while regime accuracy decreased from approximately 0.97 to 0.83. Nevertheless, it maintained better timescale and regime recovery than conventional SLDS throughout the tested range. These experiments provide practical guidance on data requirements. Under the tested settings, reliable recovery generally required observation windows sufficiently longer than the slowest timescale, adequate data within each regime, and distinguishable regime dynamics. We have included these additional results in the Appendix F.6 (Figs. 16 and 17) and discussed the implications in the revised Discussion (lines 518–529).

Q5 response: We did have ablation experiments and had mentioned them briefly in Section 2.3 in the original submissions. We have now run more controlled comparisons to investigate the contributions of these two steps. We now included these results in Appendix F.1, Fig. 8 and also explicitly referred to them in Section 3, lines 345–349 as requested.

Q6 response: This is an interesting point. We conducted additional simulations to evaluate robustness to misspecification of both latent dimension and number of regimes. First, we generated data with H=10H=10 and fitted models with H=2H=2–1616 (Fig. 14). MTS-SLDS recovered the true timescale bands more consistently than SLDS, achieving lower errors across most tested dimensions. Even when HH was overestimated, the main timescale structure remained accurate, with errors of approximately 10% and 14% at H=12H=12 and H=16H=16. Although additional spurious modes appeared, most were shorter than one sampling bin (5 ms). These sub-bin modes can be identified and excluded from timescale interpretation without removing the true modes, whose shortest timescale is 10 ms. In contrast, SLDS also produced incorrect timescale bands within the resolved range, which cannot be identified by this cutoff. We report the error in recovered true timescale bands and the number of additional spurious modes separately. Second, we generated data with K=2K=2 and fitted models with K=1K=1–33 (Fig. 15). Underestimating KK merged distinct dynamics into a single spectrum. When K=3K=3, MTS-SLDS either left the additional regime nearly unused or split an existing regime while preserving its characteristic timescales. Timescale errors remained similar at K=2K=2 and K=3K=3 (8.1% and 8.2%), whereas SLDS showed substantially larger errors. These results suggest that MTS-SLDS is more robust to moderate model overparameterization, preserving the true timescale structure even when additional latent modes or regimes are introduced. We have included these experiments in the Appendix F.5 (Figs. 14 and 15).

Q7 response: We focused on sustained fixation in V4, where the behavioral condition was relatively stationary. Previous studies also treat this dataset as one stationary process, so we adopt this setting and characterize multiple timescales within a single dynamical regime.

Q8 response: We had evaluated different numbers of regimes and latent dimensions using held-out predictive performance. The (K=2) model improved prediction over (K=1), while (K=3) did not provide consistent additional improvements. We therefore selected (K=2), which also allowed us to characterize dynamical changes associated with movement onset and relate the inferred timescales to behavior. We thank the reviewer for pointing out that these model-selection details were not clearly reported in the original submission. We have now included the evaluation procedure and results in the Appendix H, Fig. 20.

Q9 response: We thank the reviewer for this suggestion. The main difference between SLDS and rSLDS is that rSLDS allows state-dependent regime transitions. The multi-lag initialization and regime-conditioned inference ideas of MTS-SLDS could be extended to rSLDS. As requested, we also fitted Poisson rSLDS to the Area 2 data using the same data and model dimensions K=2, H=6. rSLDS achieved comparable held-out predictive likelihood to MTS-SLDS and SLDS during active reaching and slightly higher likelihood during passive perturbations. It also identified similar onset-associated regime transitions and broadly comparable timescale ranges. However, among these three methods, MTS-SLDS showed a clearer active-passive contrast in movement-regime timescales and closer correspondence with the observed behavioral decay timescales. These results suggest that MTS-SLDS provides a more behaviorally informative characterization of neural timescales. We have included these comparisons in Fig. 22, Appendix H.

Reviewer_CTihOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Multi-Timescale Switching Linear Dynamical Systems (MTS-SLDS), a framework for recovering regime-specific characteristic timescales from neural population recordings. The authors address an important limitation of traditional autocorrelation-based timescale estimation, particularly when neural dynamics are nonstationary and multiple timescales coexist within a population. Their proposed framework combines multi-lag moment initialization, which estimates the underlying dynamical operators using temporal correlations across multiple lags, with regime-conditioned Laplace Expectation-Maximization (RC-L-EM), which improves latent-state and regime inference by maintaining separate dynamical statistics for different regimes. The authors evaluate their method on synthetic datasets with Gaussian and Poisson observations, macaque neural recordings from visual cortex (V4) during fixation and somatosensory Area 2 during reaching.

Critical strengths
  1. Most conventional approaches to estimating neural timescales rely on fitting exponential decays to autocorrelation functions. These approaches can become difficult to interpret when neural activity transitions between different dynamical regimes. The proposed framework tries to address both challenges within a unified state-space modeling approach. So the paper's question is important and relatively underexplored.
  2. The distinction between accurate neural activity reconstruction and accurate recovery of the underlying dynamics is an important contribution. As shown in Fig. 2B,C, MTS-SLDS and conventional SLDS achieve similar latent reconstruction accuracy (0.82), but differ considerably in regime identification (0.95 vs. 0.54 balanced accuracy) and timescale estimation (11% vs. 44% error).
  3. Combining multi-lag moment initialization with regime-conditioned inference is a useful methodological contribution. Incorporating temporal accuracy into the initial estimation of the latent transition matrices was novel.
  4. Applying the model to biological data in addition to the synthetic data.
Critical weaknesses
  1. The methodological novelty is somewhat limited, as the framework primarily combines existing approaches. Except SLDS, the multi-lag initialization is related to what the authors also cited. Furthermore, there are alternatives that the authors did not sufficiently discuss. For example, DMD-based methods have been widely applied to spiking data recently, and they naturally give multi-timescale decomposition of the dynamics. Even simpler, given that, at the end, the authors binned spike trains, classical spectral methods could have also been an alternative approach (there is a zoo of methods available in the domain). Combining spectral methods with matrix factorization (for example, there is a wide range of NMF-based methods with solid statistical foundations), also allows the dynamics to be decomposed into different timescales and also how it evolves in time. Such things not only need to be discussed but also need to be compared. .
  2. The authors acknowledged the baseline comparisons, but did not provide more controlled comparisons. The main experiments compare MTS-SLDS with a conventional SLDS using different initialization; so, the improvements cannot be attributed independently to multi-lag initialization or regime-conditioned inference. The authors partially address this in Appendix F.1. Interestingly, for K=2K=2, both initialization methods eventually achieve approximately 9.5% timescale error when followed by the same RC-L-EM procedure, suggesting that the principal benefit of multi-lag initialization in this setting is faster convergence rather than improved final accuracy. The comparison between RC-L-EM and standard Laplace-EM uses different iteration budgets (300 vs. 100).
  3. The authors do not propose an automatic method for determining the number of dynamical regimes (states), KK. Instead, KK is specified in advance, and the algorithm identifies the corresponding regimes and estimates their timescales.
  4. The multi-lag initialization uses windowed autocorrelation descriptors followed by K-means clustering to obtain preliminary regime assignments. However, different dynamical systems can exhibit similar or identical characteristic timescales despite having different dynamical operators.
  5. If I understand correctly, the strength of MTS-SLDS is in identifying time-varying (switching dynamics) multi-timescale dynamics. However, if I'm not mistaken, all examples the authors provided for application on neural data were from simple cases. There weren’t truly complex dynamics that highlight the strength of MTS-SLDS. Perhaps the authors should have picked neural recordings from a more complex task with multiple phases and recorded from regions with rich intrinsic dynamics, like the hippocampus and PFC.
Questions for authors
  1. How much of the improvement can be attributed specifically to multi-lag initialization versus regime-conditioned inference? The authors need to provide a fully controlled comparison where the initialization, number of optimization steps, and trainable parameter sets are identical, changing only the inference procedure.
  2. The number of regimes KK is predefined, but in experimental neural recordings, the number of distinct dynamical regimes is typically unknown. How does the method behave when the specified KK differs from the true number of regimes? Does it split one regime into several states or merge dynamically distinct regimes? How sensitive will conclusions be to inference about the number of states from data?
  3. Can the framework distinguish dynamical regimes that have similar timescale spectra but different dynamical structures?
  4. How does the computational cost scale with the number of neurons, latent dimensions, and dynamical regimes?
  5. As noted in points 1 and 5 of the weaknesses above, how can MTS-SLDS be compared to a wider range of methods (beyond the SLDS family), and how it perform with complex recordings and tasks?
4 comments
AuthorsOpenReview ↗

We thank the reviewer for these helpful comments.

W1 response: MTS-SLDS uses the standard SLDS generative model but introduces a two-stage learning procedure aimed at improving the recovery of regime-specific timescales and switching structure. Although multi-lag moment methods have been developed for stationary LDS, our approach extends them to switching dynamics through preliminary regime segmentation and regime-specific moment estimation. RC-L-EM builds on existing literature but uses regime-conditioned state and across-time moments with Laplace approximations for Poisson observations to refine the dynamical parameters.

We appreciate the suggestion to consider DMD and NMF. We have expanded the related-work discussion to clarify the connections and differences between these approaches (lines 111–127). Although these methods are useful for characterizing temporal patterns in neural activity, their standard formulations do not jointly model latent switching dynamics and Poisson spike observations as in MTS-SLDS. We also conducted additional synthetic experiments using delay-DMD for stationary and switching dynamics, providing DMD with the true regime segmentation in the switching comparison. Its timescale estimation accuracy deteriorated as observation noise increased (Fig. 12).

W2 response: We agree that the original comparison (Appendix F.1, Fig. 7D) did not fully isolate the contributions of initialization and inference because the optimization budgets and parameter updates differed. We therefore conducted a fully controlled comparison using identical initial parameters, trainable parameter sets, and 300 iterations for both inference procedures (Fig. 8). We tested both multi-lag moment initialization and standard SLDS initialization on 10 Poisson datasets with K=2K=2.

Results (median timescale MAPE / held-out regime accuracy):

InitializationRC-L-EMStandard Laplace-EM
Multi-lag moment9.6% / 0.9534.0% / 0.65
Standard SLDS9.6% / 0.9339.2% / 0.57

RC-L-EM consistently improved both timescale and regime recovery, outperforming standard Laplace-EM on all 10 datasets from either initialization. Multi-lag initialization also contributed: at K=2K=2, it accelerated convergence (10 vs. 130 iterations to reach 15% error; Fig. 7A), while at K=3K=3, it improved regime recovery, with no regime-loss failures compared with 6/10 datasets using standard initialization (Fig. 7B,C). Finally, increasing standard SLDS optimization from 100 to 500 iterations did not improve its timescale recovery (median errors of 32.9%, 35.9%, and 36.1% at 100, 300, and 500 iterations; Fig. 9). These controlled results demonstrate that regime-conditioned inference provides the main improvement in final timescale accuracy at K=2K=2, while multi-lag initialization improves convergence and robustness at higher KK.

W3 response: Determining the number of dynamical regimes K is a general model-selection challenge shared by SLDS and related switching models. MTS-SLDS also allows K to be selected through standard model-selection procedures, such as comparing held-out predictive likelihoods across candidate values. Our work focuses on accurate recovery of regime-specific timescales and switching structure. We actually had performed model selection for K on the Area 2 dataset and now provide the detailed evaluation procedure and quantitative results to justify our choice of K=2 (Appendix H, Fig. 20).

W4 response: We thank the reviewer for this interesting question. Although the initial autocorrelation-based segmentation may not distinguish regimes with identical timescales, RC-L-EM uses the full dynamical structure to refine regime identification. To test this, we conducted additional simulations with two regimes having identical eigenvalue spectra but different dynamical operators related by an orthogonal rotation (A2=SA1S⊤A_2=SA_1S^\top). Across 10 Poisson datasets, MTS-SLDS achieved substantially higher median regime accuracy than conventional SLDS (approximately 0.96 vs. 0.79; Fig. 13). These results demonstrate that MTS-SLDS can distinguish regimes with identical timescales but different dynamical structures.

W5 response: We agree that applying MTS-SLDS to more complex neural dynamics would further demonstrate its capabilities. While the original submission focused on relatively small numbers of regimes, we have conducted additional synthetic experiments with larger K (up to 5) to evaluate the method's scalability (Appendix F.6, Fig. 17). Moreover, we have applied MTS-SLDS to an additional neural dataset from NLB by fitting three dynamical regimes, with the neural activity recorded from the dorsomedial frontal cortex (DMFC) of a macaque performing a time interval reproduction task. Preliminary results suggest that MTS-SLDS can identify distinct regime-specific timescales and provide more interpretable estimates than SLDS. We have included these results in Appendix I, Fig. 24 as an initial demonstration. A more systematic evaluation on more complex tasks, like neural data in hippocampus and other regions, will be one important direction for future work.

Q1 response: As detailed in our response to W2, we have conducted a full controlled comparison using identical initial parameters and same optimization iterations. RC-L-EM reduced timescale error from 34.0% to 9.6% and improved regime accuracy from 0.65 to 0.95. These results show that regime-conditioned inference contributes most to final accuracy at K=2, while multi-lag initialization mainly accelerates convergence and also improves robustness at higher K (Appendix F.1, Fig. 8).

Q2 response: We conducted additional experiments with two true regimes, fitting models with K=1,2,3K=1,2,3 (Fig. 15). When K=1K=1, the model merged the two distinct dynamics into a single spectrum, resulting in 31–37% timescale error. When K=3K=3, MTS-SLDS either left the extra regime nearly unused (2/5 datasets) or split an existing regime into two (3/5), with both fitted regimes preserving the original timescales. Consequently, timescale error remained nearly unchanged between K=2K=2 and K=3K=3 (8.1% vs. 8.2%), while SLDS showed substantially larger errors. Held-out prediction improved considerably from K=1K=1 to K=2K=2 (+27 nats/trial), but only slightly from K=2K=2 to K=3K=3 (+3.6 nats/trial). Similar diminishing improvements were observed on Area 2 (Appendix H, Fig. 20). These results suggest that MTS-SLDS is relatively robust to moderate overestimation of KK. Based on these, we recommend examining held-out prediction gains, regime occupancy, and timescale stability across neighboring values of K when selecting the number of regimes in real application.

Q3 response: As detailed in W4, additional simulations show that MTS-SLDS can distinguish regimes with identical eigenvalue spectra but different dynamical operators, achieving approximately 0.96 regime accuracy compared with 0.79 for conventional SLDS (Fig. 13).

Q4 response: We analyzed computational complexity and benchmarked runtime across the number of neurons DD, latent dimensions HH, and regimes KK. The per-iteration complexity is O(T[KNN(DH2+H3)+K2H3])O(T[KN_N(DH^2+H^3)+K^2H^3]), with empirical runtime scaling approximately as D0.96H1.95K1.10D^{0.96}H^{1.95}K^{1.10}. MTS-SLDS requires approximately 0.8-2.2 times more computation per iteration than SLDS for K=1K=1–33. These results are included in Appendix Appendix C.6, Fig. 5.

Q5 response: We note in original submission, we had multiple other baseline methods compared as MTS-SLDS, such as tradition ACF-based and aABC on V4, and on area2, in addition to SLDS, we also included a method sPCF for comparing timescale estimates on Area2. As discussed in W1 and W5, we now added additional comparisons with delay-DMD on synthetic data and experiments on a three-regime neural task. We additionally compared with rSLDS on Area 2, finding comparable predictive performance but weaker correspondence with behavioral timescales than MTS-SLDS. We also investigated data requirements by varying trial length and the number of regimes K=1-5, which shows recovery remained reliable with sufficient observations. These additional results are presented in the Appendices F.3, F.6, H, and I (Figs. 12, 16, 17, 22, and 24), with the observed limitations discussed in the revised manuscript.

I thank the authors for the rebuttal and additional analyses. The synthetic experiments with larger K indeed show the potential strength of the proposed method in more complex settings. However, the current results (including new DMFC data in the ready-set-go task) still do not demonstrate how this strength can be relevant for providing new insights into complex real neural dynamics without predefined regime changes. The ready-set-go task is still relatively simple with labeled task epochs, and new DMFC analyses are only on a small subset of data without confidence intervals. With this analysis, it is difficult to evaluate whether differences between SLDS and MST-SLDS are significant. Using labeled task epochs, one could also use autocorrelation-based methods. For a methods paper, it is essential to demonstrate its application on real data with the kind of complexity the method is designed for.

Reviewer_cZb3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents MTS-SLDS, a variant of SLDS models that improves the ability to estimate the timescales of dynamics within different states, relative to standard SLDS models. It does so using 2 main updates to a standard SLDS - to the initialization of the dynamics and to the inference method. While neither is that novel on its own, the combination leads to meaningful improvements, so I believe is a valuable contribution overall.

Critical strengths

Benefits versus standard SLDS methods are shown within a simulation and multiple real datasets. I found the ablation studies to be very valuable for understanding the reasons behind the differences.

Critical weaknesses

-I found the background hard to follow (appreciate them being thorough with respect to mentioning a lot of literature, but just mentioning many different approaches with limited description of each of them made it hard to follow, and hard to understand what aspects of your approach are similar and different to past methods)

-Minor: the figure 1 caption ends in the middle of a sentence

-Very minor: Abstract says “area 2 motor cortex”, but it’s somatosensory cortex

Questions for authors

-The figure 3 results make me question things. Why is there a difference between models when there is only 1 state? Shouldn’t the standard laplace EM be almost identical to your RC-laplace EM when there is only 1 state? Clarification on this would be re-assuring.

-What is the link function used in the SLDS model from the ssm package (softplus, poisson, poisson_orthog, etc)?

-What do the states in Fig. 2 correspond to in the standard SLDS? Is it splitting the two states based on active/passive instead of based on when the monkey is moving?

6 comments
AuthorsOpenReview ↗

We thank the reviewer for these helpful comments.

Weaknesses response: We have revised the Introduction to provide clearer explanations of the most relevant existing methods and to better highlight their similarities to and differences from our approach (lines 97–128). The two typos are corrected, and we also carefully checked the manuscript for other typos.

Q1 response: We agree that when K=1, the distinction arising from regime conditioning may disappear, since there is only one dynamical regime. However, the two methods can still differ in their initialization and approximate inference procedures. Specifically, MTS-SLDS uses multi-lag moment initialization to capture temporal structure across multiple lags. In the synthetic Poisson experiments K=1, SLDS and MTS-SLDS achieve comparable timescale recovery, with small differences in estimation accuracy (Fig. 2A). Similar differences can also arise in the V4 recordings, even with a single regime. In addition, the model-generated autocorrelations from MTS-SLDS more closely match the empirical autocorrelations (Fig. 3D), which suggests that its fitted dynamics capture the observed temporal dependencies more accurately.

Q2 response: The SLDS baseline uses the exponential link function as same as MTS-SLDS. We now specify it clearly in the corresponding Appendix D.4.

Q3 response: We assume the reviewer is referring to Fig. 4. Both SLDS and MTS-SLDS were fitted separately to the active and passive conditions. Therefore, the inferred discrete states reflect within-condition dynamical regimes, not a separation between active and passive conditions. In both methods, the inferred regime transitions are associated with movement onset of behavior.

Some follow-up questions

Could you further clarify the Q1 response, as it's still not intuitive to me how with a single discrete state, there is such a difference between MTS-SLDS and SLDS in Fig 3B -How is the held-out likelihood computed in 3B? -What is different about the RC-Laplace-EM versus Laplace-EM inference when there is only one discrete state? -In practice, am I understanding correctly that the multi-lag moment initialization will still yield a single dynamics matrix based on one lag (the MTS-SLDS is still an AR1 model)? How are things initialized in the ssm SLDS package you're using as a baseline? And you think this dynamics matrix initialization may be the reason for the difference?

For Fig 4 (area 2 data), are the learned states almost identical for the MTS-SLDS and SLDS (divided based on movement), as it sounds like you're implying?

Ultimately, both these questions are trying to help me understand where the advantages of MTS-SLDS are coming from - are benefits at all related to MTS-SLDS learning different discrete states, or they are driven by more time-scale-relevant dynamics within discrete states?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.