- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes ML-DLDS that learns interpretable dynamics across multiple lags. ML-DLDS models interaction via linear mappings with lag gains, time-varying coefficients. The model is optimized by assuming structured sparsity on lag gains and coefficients . The proposed method is evaluated on synthetic datasets, Neuropixels recordings and COVID time series. Overall, I think the neuro-science motivation is good, but the contribution appears to be an incremental extension of decomposed linear dynamical systems with limited technical novelty and insights. Experimental results are not very convincing in that it does not show meaningful structures recovered and why/when ML-DLDS is better than the existing methods.
Critical strengths
The neuro-science background of the paper is appreciated: I agree that understanding how multiple neural populations interact across different timescales is an important problem. The authors have done a good job in positioning the paper among the literature. Technically, sharing interaction matrices across delays is a sensible modeling choice that could provide a interpretable description of these interactions. Experimental results show that to some extent incorporating multiple delays is useful.
Critical weaknesses
I am afraid the paper has limited novelty overall. ML-dLDS is based on dLDS, which already represents dynamics through sparse combinations of shared interaction matrices. Compared against existing methods, the main contribution of the paper seems a sparse lag profile for simultaneously active shared circuits. However, the paper provides little technical insight into what advantage ML-dLDS offers over the existing methods. In the related work section, these methods are discussed but not evaluated experimentally. I also find that the synthetic datasets seem to match the assumptions. For hard problems, these assumptions do not necessarily hold.
Questions for authors
- what is the main advantage and improvement over existing methods like dLDS besides allowing multiple lags? Does ML-dLDS provide new insights beyond the baselines?
- is it possible to compare against stronger methods that already model multiple delays or nonlinear interactions?
- do the neural findings hold across additional recording sessions or animals? I notice that only one session is used in the Neuropixels dataset. How representative is this?
2 comments
Rebuttal, part 2, Dfp8
W3 and Q2. Comparison to related methods.
We thank the reviewer for emphasizing the importance of strong baselines, and we fully agree that there are always further methods one could compare to, including all those we mention in the related work. In the original submission we compared ML-dLDS against four to six baselines across its experiments (App. C, Table 2, Table 3). When designing the experiments, we focused on baselines that produce components equivalent to those of ML-dLDS, such that the comparison can show whether ML-dLDS recovers the underlying circuits, dynamics, and lags less/more accurately than they do.
Regarding the reviewer’s question on the possibility of comparing against methods that model multiple delays or nonlinear interactions, these methods do not provide components analogous to ours, which makes a comparison ill-defined. For example, a comparison to LFADS (Pandarinath et al., 2018) could be made on data fidelity, but it would not allow a proper comparison of the latent dynamics / circuits, as LFADS does not learn a set of circuits analogous to ours (Sec. 1).
Multi-lag methods like mDLAG (Gokcen et al., 2023) would not address the comparison of the dynamical process we focus on either, as mDLAG is not tailored to learn the dynamical operator that explains the temporal evolution. mDLAG instead models each brain area as a combination of a shared set of Gaussian-process latents that appear with a different delay in each area. That means that also the definition of a lag in mDLAG differs from our definition, as a lag in mDLAG is the time offset at which a shared latent appears in each area, whereas in our model a lag is how far back in time a circuit (i.e., the dynamical operator that governs the latent dynamics) reaches to drive the present state (Sec. 2). In addition, the delay in mDLAG is a single fixed value per latent and per area that stays constant over the recording, whereas in our model the lags at which each circuit acts vary in time through the coefficients. The two lags thereby capture different quantities, and only the cross-area part of our lags has a counterpart in mDLAG, as mDLAG has no notion of a circuit that acts within an area or across several lags. We therefore do not necessarily expect to observe an agreement between them.
If the reviewer has a specific method in mind that they believe produces analogous components and should have been included, we would be happy to add it to the revised paper.
W4. Synthetic data and the model assumptions.
We thank the reviewer for noting that the synthetic datasets match the model's assumptions, and we agree this is an important test. We therefore added three new experiments, each across five seeds, 1) synthetic data in which every circuit acts at the same single lag, 2) data generated from a vector autoregression with a separate matrix per lag, and 3) data generated from nonlinear delayed dynamics.
For the first, we tested whether ML-dLDS recognizes the single lag or instead spreads the dynamics across longer lags, and found that it kept lag one as the dominant lag of every circuit (Fig. 15). For the second and third, we tested whether ML-dLDS recovers the correct lags on VAR-driven and on nonlinear data, and found that it recovered the dominant lag of every circuit on both, with a lag-profile correlation of on the VAR data and on the nonlinear data (Fig. 17).
Q3. Additional sessions.
We agree that an additional neural data session could further strengthen our paper. The data are large and require compute access that is hard to get within a short rebuttal period. We are working on it and will include the additional session in the revised paper.
We hope these clarifications and the new experiments address the reviewer’s concerns and questions.
Rebuttal, part 1, Dfp8
We thank the reviewer for the careful reading and for recognizing the value of modeling multi-lag dynamics. We address each point below.
W1 and Q1 (part a). Main advantage and improvement over existing methods.
We thank the reviewer and agree that dLDS is the foundation of our model. We claim, however, that ML-dLDS is not a straightforward extension of dLDS to multiple lags. In particular, a naive multi-lag extension of dLDS would suffer from the natural ambiguity of the system, as its many parameters admit multiple equivalent solutions when past states at different lags carry similar information. We kindly refer the reviewer to the original submission, where we discussed this inherent ambiguity (Sec. 3.2, App. J). This challenge requires a new model that copes with this ambiguity via structured regularization, and accordingly also requires a modified inference procedure. The novelty of ML-dLDS is therefore fourfold: 1) a lag matrix S with structured regularization, 2) a modified inference procedure, 3) a new initialization, and 4) recovery of the lags at which each circuit acts, which a standard dLDS would obscure and simplify into a single lag.
For the first, the lag matrix S assigns each circuit a sparse set of lags, and its structured penalty selects among the equivalent solutions (Eq. 4). For the second, the inference accounts for the coupling of each circuit across its lags when it updates the coefficients, the lag gains, and the circuits (Sec. 3.3). For the third, the initialization detects segments dominated by a single circuit and reads the lag of each circuit from them (App. A). For the fourth, we demonstrated the advantage empirically on multiple experiments. For example, on the mouse recordings ML-dLDS finds interactions from the lateral posterior thalamus at δ=1 that re-emerge at δ=4 (Fig. 4D), and on the COVID-19 data short delays dominate the synchronized waves while longer delays emerge in the desynchronized phase (Fig. 5E).
We also kindly refer the reviewer to the experiments in our paper where we actively tested this effect. In particular, when we removed the lag penalty on synthetic data, the lag-support recovery dropped sharply (Fig. 3G, Table 2). In addition, single-lag dLDS reached a much lower circuit correlation than ML-dLDS (Table 1). We believe that these changes are enough to make ML-dLDS a standalone model.
W2 and Q1 (part b). Technical insight into the advantage.
We appreciate the reviewer's recognition that more clarification and technical insight into the advantage of ML-dLDS would improve the paper. Following the reviewer's concern we clarify the mechanism behind this advantage. When the true dynamics act at lags larger than one, a model limited to a single lag would absorb the earlier states into its operator. For example, if the true dynamics are xt=fxt−2, the best single-lag fit xt ~ Mxt−1 gives M=fC21C11−1 (for simplicity, assuming centered, stationary data), where C21 is the covariance between xt−2 and xt−1 and C11 is the covariance of xt−1. The recovered operator therefore mixes the true circuit f with the state's own autocorrelation. These mixed operators will differ from the true circuit and will not allow us to know at which lag the circuit acts. In contrast ML-dLDS explicitly learns the lag at which each circuit operates and recovers circuits that are closer to the true ones on synthetic data (Table 1, App. J). With respect to VAR specifically, VAR does model multiple lags but learns one fixed matrix per lag. That means that it cannot capture multiple co-occurring processes at a time and cannot reuse the same operator across multiple timescales. Consequently, ML-dLDS can capture richer dynamics with a comparable number of unknown parameters.