acceptodds
Under review as a conference paper at ICLR 2027

State and Flow: Differentiable Switching State Space Models for Dynamical Systems Reconstruction

Abstract

Dynamical systems reconstruction (DSR) leverages neural surrogate models to recover underlying dynamics from noisy, indirect observations. Complex real-world systems often exhibit both continuous nonlinear flows and discrete regime switching. However, existing DSR approaches tackle these challenges in isolation: Neural Ordinary Differential Equations (NODEs) capture expressive nonlinearities but lack discrete states, whereas recurrent Switching Linear Dynamical Systems (rSLDS) identify discrete regimes but restrict per-regime dynamics to linear transformations. Moreover, while DSR surrogates are intended to be simulated and perturbed, current training and evaluation typically reward reconstruction rather than autonomous generation. To bridge these gaps, we introduce a latent autoencoder framework that emphasizes autonomous rollout generation during training, along with two novel architectures: recurrent differentiable SLDS (rdSLDS), an end-to-end trainable rSLDS, and recurrent Switching Neural ODE (rSNODE), which combines discrete state inference with nonlinear NODE flows. Both models infer discrete states in an unsupervised manner. To evaluate models on systems exhibiting coupled nonlinear and switching behavior, we propose the switching Lorenz system as a new benchmark. Across chaotic, piecewise-linear switching, and chaotic switching benchmarks, rdSLDS recovers the ground truth linear regime structure but rarely converges under chaotic switching, where rSNODE alone reconstructs both the attractor geometry and the correct two-state segmentation. Applied to task-fMRI data, rSNODE generates complete regional time series from short initial windows, preserves their temporal and task-activation structure, and identifies states aligned with broad experimental conditions without using task labels. Combining interpretable switching regimes with the expressive power of NODEs yields a scalable, versatile tool for DSR in neuroimaging and beyond.

Primary area:
learning on time series and dynamical systems
Keywords:
dynamical systems reconstruction, switching linear dynamical systems, neural ODEs, state space models, time series, unsupervised learning, interpretability, computational neuroscience

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kHsiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces rdSLDS, which uses Gumbel-Softmax gating over linear transitions, and rSNODE, which replaces these linear dynamics with regime-specific neural ODEs to increase capacity to model complex nonlinear dynamics. models generate trajectories autonomously after initialization from an observation window. The evaluation includes Lorenz dynamics, the piecewise-linear NASCAR system, a newly proposed switching-Lorenz benchmark, and task fMRI data.

Critical strengths
  1. Developing an end to end differentiable formulation of switching linear dynamical systems is well motivated.
  2. The switching Lorenz benchmark provides a valuable setting for evaluating models that must capture both nonlinear within regime dynamics and transitions between regimes.
  3. The emphasis on autonomous trajectory generation is meaningful.
Critical weaknesses
  1. The paper should more clearly distinguish the contribution of the Gumbel–Softmax relaxation from conventional discrete-state inference in rSLDS. The results visualize a single inferred regime at each time step, whereas the model equations describe a soft mixture of K regime-specific transitions. Are these visualizations obtained by taking the argmax of the mixture weights? During generation, does the model retain the soft mixture or use a hard regime assignment? Clarifying this distinction would help readers understand how relaxation affects the learned dynamics and the interpretation of the inferred states..
  2. Does the model use any initialization method for the switching dynamics? Or are the switching parameters randomly initialized? Since switching models can be sensitive to initialization, a discussion of its effect on convergence and regime recovery would be useful.
  3. The baseline comparisons are insufficient to establish the advantages over existing switching models. In particular, a direct comparison with Linderman et al.’s rSLDS is needed to assess the benefits of the proposed differentiable formulation..
  4. Because recovering meaningful discrete regimes is a central contribution, regime recovery should be evaluated quantitatively.
  5. The quantitative tables could be presented more clearly. Highlighting the best result for each metric, e.g., using bold, would make comparisons easier to interpret.
  6. In section B.7.1, the authors state ‘Since MMD operates on the full multivariate trajectory, it captures differences in both marginal distributions and temporal correlations between true and generated dynamics’. However, MMD measures two distribution instead of temporal correlation, the formulation in Equation 19 compares distributions of multivariate observations without explicitly incorporating temporal ordering.

Overall, I appreciate this paper’s idea on autonomous generation instead of only reconstruction. However, several weaknesses should be addressed to strengthen the paper.

Questions for authors

Please see weakness.

2 comments

We thank the reviewer for the constructive review and address each point below. Please also see our general response.

Gumbel-Softmax and hard state generation

rSLDS keeps discrete states and infers them by approximate posterior inference (Laplace-EM in ssm). We relax the discrete state with Gumbel-Softmax so the whole model can be trained by backpropagation through rollouts. Training uses the soft mixture (temperature annealed from 1.0 to 0.1); generation uses hard one-hot selection, so the visualized regimes are the model’s own outputs, not a post-hoc argmax (Sections 3.1–3.3). Since soft mixtures can combine vector fields (Poli et al., 2021), we report regime occupancy, which shows whether free runs visit the regimes as often as the true system (Table 6).

Initialization of switching dynamics

Regime dynamics are randomly initialized (ODE weights scaled by 0.5; rdSLDS matrices near the identity; gate with default initialization), without pre-clustering. We report the effect across 15 seeds: failures per model and K (Table 5) and the spread of D_stsp and ARI (Tables 5, 6). rSNODE fails on at most one seed, whereas rdSLDS is sensitive to initialization on switching Lorenz (2, 9 and 11 failed seeds for K = 2, 4, 8).

Comparison to Linderman’s rSLDS

We added rSLDS, fitted and evaluated with the same pipeline (general response, point 4; Appendix C.2, D.5). On the Lorenz systems, rSLDS generally outperforms rdSLDS, which likely reflects its inference of a posterior over the full latent trajectory, compared with rdSLDS’s initial-state inference and rollout-based training; we therefore present rdSLDS as the linear-regime variant of our framework. Since nonlinear switching models trained end-to-end already exist (e.g. SNLDS), we do not attribute rSNODE’s gains to the relaxation itself, which mainly enables training on rollouts. rSNODE reaches a lower D_stsp than rSLDS on chaotic and noisy Lorenz; on switching Lorenz, rSLDS has a competitive D_stsp but diffuse, barely or not chaotic free runs with a higher D_H. On NASCAR, a piecewise-linear system, the linear switching models perform best: rdSLDS on D_stsp, rSLDS on D_H and regime recovery (Tables 5, 6).

Quantitative evaluation of regime recovery

We now quantify regime recovery with the ARI in three versions (gate, free run, and attractor as main measure), together with on-attractor share, coverage and occupancy (Appendix C.1.6, Table 6); details are in our response to Reviewer 8dqf. On switching Lorenz, rSNODE (K = 4) achieves the highest attractor ARI (0.37, vs. 0.21 for rSLDS and at most 0.16 for rdSLDS) and reproduces the true occupancy, but recovery is partial for all models, and we revised our claims accordingly. On NASCAR, rSLDS and rdSLDS identify the regimes best.

Table presentation

All tables now mark the best result per metric in bold and include ground-truth rows and failure counts. We would be happy to improve any other part that is hard to read.

MMD metric

Correct; we fixed the description: MMD on single states compares state distributions and ignores temporal order, as does D_stsp, while D_H captures temporal structure (Appendix C.1.4).

We hope these revisions address your concerns and welcome further questions.

I thank the authors for addressing my concerns. I now have no further questions and will discuss more with the AC and other reviewers before adjusting my score.

Reviewer_8dqfOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses dynamical systems reconstruction (DSR), i.e. learning latent dynamical systems from time series data. It focuses on the case of switching state-space models that are composed of discrete latents which model the switching between different continuous latent dynamics ("switching dynamical systems"). The authors present two DSR models to address the challenge. Both models are based on variational autoencoder frameworks with a bidirectional RNN encoder that encodes observed trajectories to latent initial conditions for the continuous latent state (regime dynamics) and the discrete latent state (regime indicator). Further inference does not access the data and a linear decoder is used to get back to observation space. The authors claim that this targets autonomous generation more so than previous DSR approaches, and that their approach can infer the discrete latents in an unsupervised manner.

The first presented model is called recurrent differentiable switching linear dynamical systems (rdSLDS). This is an rSLDS-like model [Linderman et al., 2017] with an MLP-parameterized gate that selects Gumbel-softmax-weights of a mixture distribution of affine maps for each of the KK regimes. The second model is called recurrent switching neural ODE (rSNODE). Here, each affine map of rdSLDS is replaced with a neural ODE, hence each regime can exhibit nonlinear dynamics. The authors include ordinary ("single-regime") latent NODEs as the baseline.

The models are validated on the 3d chaotic Lorenz-63 system, a four-regime limit cycle ("NASCAR") with 10d-observations, a newly introduced "switching Lorenz" system that switches between two Lorenz-63 attractors, and on real-world task-conditioned fMRI data ("EMOTION" and "MOTOR"). To assess performance, the authors mainly use a kernel maximum mean discrepancy (MMD) metric for autonomous generation fidelity (attractor reconstruction) and the (estimated) maximum Lyapunov exponent (MLE). Power spectrum correlation (PSC) and MSE are also reported in the supplement.

After a sanity check of all models on the Lorenz-63 system, the authors claim that rdSLDS recovers the four NASCAR regimes unsupervised but mostly fails on the switching Lorenz where there are 2 nonlinear/chaotic regimes, and that rSNODE with K=2K=2 regimes reconstructs this system well. On fMRI, the authors claim that rSNODE states align with task blocks without using the true labels.

The authors state that the choices of linear-or-nonlinear regimes (rdSLDS/rSNODE), the amount of capacity of each ODE in the latter and the choice of KK have to be defined by the user of these models. Further, rSNODE gives arbitrary segmentations (indistinguishable performance for multiple KK) on the two simpler benchmarks (Lorenz-63, NASCAR).

Critical strengths
  • I view "switching-dynamics DSR" as an important challenge to address. I imagine that multiple real-world applications (e.g. the shown fMRI data) could benefit from an advance in that field. The introduced switching Lorenz system seems like a nice idea; this benchmark and further extensions of it may help with validating those DSR models.

  • The authors transparently report their (15-seed) runs alongside hyperparameter choices and parameter counts. They include a limitation paragraph that discusses some of the main issues with the presented models.

Critical weaknesses

I found several weaknesses and discrepancies in the paper. Since the reviewing form suggests it, I will keep the weaknesses to my major points within 2 subjects.

  • First, the central empirical claims of the paper do not seem to be supported by the shown evidence.

    • The abstract, caption of Fig. 4 and discussion state that rSNODE outperforms both non-switching NODEs and rdSLDS. But the p-values shown in Fig. 4G are all non-significant (except for NODE vs. rdSLDS (K=2K=2), where rdSLDS is worse) per the authors' own significance level; and the authors do not test rdSLDS vs. rSNODE at all.
    • The implementation of the primary evaluation metric (MMD) does not fulfill its stated purpose (expressing distribution divergence, so attractor geometry reconstruction for dissipative/chaotic systems). The sensitivity with respect to kernel bandwidths is not reported. The trained models in Table 4 that are described as successful are between 0.0170.017 and 0.0780.078 for the Lorenz-63 attractor. However, in Fig. 9a, rdSLDS with K=1K=1 scores around 0.040.04. This is a single affine map, it cannot resemble anything that is close to the bounded chaotic Lorenz attractor, but it lies just in range of the trained models (rdSLDS with K=2K=2 has 0.0460.046). Hence the applied MMD metric doesn’t discriminate between a model which is structurally incapable of producing chaos and models claimed to reconstruct the chaotic Lorenz-63 well. Moreover, the authors state in B.7.1 that MMD "captures differences in both marginal distributions and temporal correlations between true and generated dynamics". In Eq. 19 however, with the Gaussian kernel chosen by the authors, I cannot identify anything that would capture temporal correlations.
    • The applied Rosenstein method to estimate the model's largest Lyapunov exponent (MLE) seems too noisy to support claims about matching the reference. This method is also known for its sensitivity with respect to the choice in hyperparameters. For the Lorenz-63 system, the ground truth largest Lyapunov exponent is known to be at around 0.9060.906 (in continuous time). In Table 4 the across-seed deviations of MLE on Lorenz-63 are up to ±\pm 0.526 (for rdSLDS). Also, the authors state in B.7.2 that "a positive MLE confirms that the reconstructed dynamics are chaotic". This is not the case, even if the MLE is estimated precisely, diverging unstable dynamics will also have a positive leading exponent.
    • In Figure 5D, the authors report a linear fit with r=0.415r = 0.415. That means r2=0.17r^2 = 0.17, so only 17%17\% of the variance in the learned model's regions of interest (ROI)-task correlations are explained by the data. Further, the slope of the fitted line implies a mismatch in the learned correlations (roughly 3×3\times). It is not clear to me how the conclusion in the caption that the ROI-task correlations "match the true data" follows from this (single-subject) fit.
    • One of the main stated advantages is unsupervised recovery of the discrete latents (i.e. regimes). For NASCAR and switching Lorenz, the trajectory of ground truth discrete labels exists, but there is no score reported that measures this against the models. The authors' 3-seed figures in that regard show no errorbars (e.g. Figure 3F) and are based on the MMD metric whose values seem to be not discriminative for reconstruction fidelity (see above).
  • Second, I have significant concerns regarding the way existing literature is presented and about the novelty claims in the paper.

    • One of the main claims is that the authors' approach is focused on autonomous generation more so than existing approaches, since they "emphasize autonomous rollout generation during training". However, in the presented model, the bidirectional encoder is trained to reconstruct the same 50-step window it has seen. With the authors' discretization Δt=0.01\Delta t = 0.01 this amounts to under half a Lyapunov time for the Lorenz-63 system. The model thus aims at reconstruction of a short-time window from a smoothed/data-conditioned initial state. This approach is a known remedy for exploding gradients in (chaotic) DSR [Mikhaeil et al., 2022], and it also explains the failure at Δt=0.1\Delta t = 0.1 that is reported in Appx. A. In general, I would not agree that the inference being data-conditioned means that the learned generative model cannot be run autonomously or is not "truly generative". Otherwise, one could argue the same for the observation-encoding VAE frameworks that the authors present here.
    • The text states that current training and evaluation in DSR "reward reconstruction rather than autonomous generation", and the introduction cites [Koppe et al. (2019)] for models that "encode the full observation sequence at both training and inference time, so they never act as true generative surrogates". However, in Koppe et al. (2019), the authors' stated aim is a latent model that captures the dynamics when run autonomously; they introduce both a KL-divergence measure on state space occupancy and power spectrum agreement using freely generated model trajectories to test exactly that. In fact, free-running/autonomous evaluation is standard practice in recent DSR methods, as done in the work that the authors cite themselves, e.g. Kramer et al., 2022; Durstewitz et al., 2023; Brenner et al., 2025; and beyond it [Pathak et al., 2017; Li et al., 2022; Jiang et al., 2023; Schiff et al., 2024].

References

  • Mikhaeil, et al. On the difficulty of learning chaotic dynamics with RNNs, NeurIPS, 2022.
  • Pathak, et al. Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data, Chaos, 2017.
  • Li, et al. Learning chaotic dynamics in dissipative systems, NeurIPS, 2022.
  • Jiang, et al. Training neural operators to preserve invariant measures of chaotic attractors, NeurIPS, 2023.
  • Schiff, et al. DySLIM: Dynamics stable learning by invariant measure for chaotic systems, ICML, 2024.
Questions for authors
  • Can the authors clarify whether their reported p-values (especially in Fig. 4G) are Bonferroni-corrected (as stated in the Appx. B.7.4) or not; and can they also provide p-values for comparisons of rdSLDS against rSNODE?

  • Can the authors evaluate their MMD metric between two trajectories drawn from the reference Lorenz system (with the same settings, e.g. trajectory length TT, number of evaluated points, and bandwidths) to establish a "baseline" for this metric? Also, can the authors report how sensitive their MMD scores are with respect to the choice of bandwidths? In Schiff et al., 2024, the authors note that they avoid MMD with Gaussian kernels because of the sensitivity to those hyperparameters. Finally, can the authors report standard measures used in DSR for autonomous/long-term dynamical fidelity, e.g. state-space occupancy DstspD_\mathrm{stsp} and Hellinger distance between power spectra DHD_H from the cited DSR literature, e.g. Koppe et al., 2019; Volkmann et al., 2024.

  • Can the authors report an agreement score on the unsupervised recovery of discrete regimes, measuring ground truth labels against reconstructed ones? Adding this for NASCAR and switching Lorenz would clarify one of the main stated advantages of the presented models. Also, since only the latent initial conditions are inferred from data and all following states are from autonomous rollouts, can the authors clarify how the inferred regime sequence is aligned with respective ground truth sequences for this comparison (e.g. per time step of the freely-generated rollout)?

3 comments

We thank the reviewer for the thorough and expert review. We agree with most of the concerns and have made substantial changes, summarized in the general response. We share the reviewer’s view that switching dynamics is an important open challenge for DSR, and we hope that the revised evaluation, together with the switching Lorenz benchmark, makes this work a useful contribution to the DSR community. We further address each point below.

Part 1: Support for the empirical claims

Significance tests and p-values

The p-values in Fig. 4G were uncorrected and compared against α = 0.05/6; the appendix wrongly described them as corrected, which we fixed. By this criterion, rSNODE did not significantly outperform NODE on MMD, and rdSLDS vs. rSNODE was not tested. Given the issues with MMD pointed out by multiple reviewers, we repeated all tests on held-out data with the revised metrics, including rdSLDS vs. rSNODE on matched K (Table 7), and revised all claims accordingly (general response, point 2). rSNODE additionally provides a discrete segmentation, which NODE cannot; we now quantify how well it recovers the true regimes (Table 6), which is partial.

Evaluation Metrics We agree. Because the MMD bandwidth was estimated from pooled true and generated samples, it changed between models and could mask poor reconstructions, consistent with the reviewer’s observation on Fig. 9a. As suggested, we now report D_stsp and D_H (Durstewitz lab implementation) as primary metrics. As a baseline, every metric is computed for true-system trajectories with the same protocol (GT rows, Table 5). MMD now uses a bandwidth fixed per reference and is reported for three bandwidths, showing its bandwidth sensitivity; we note the concern of Schiff et al. (2024). We corrected the claim that MMD captures temporal correlations. We also found that D_stsp favours diffuse free runs (Appendix C.1.1, Figure 9), so we interpret it together with D_H and λ₁.

MLE estimation

We now compute λ₁ from the Jacobians of the learned dynamics (Appendix C.1.3); on the true equations this gives 0.905 (Lorenz-63) and 1.01 (switching Lorenz). On Lorenz-63, NODE and rSNODE (K = 1) match closely (0.87, 0.90), while rSNODE with K = 2, 4 underestimates (0.72, 0.78); on switching Lorenz, rSNODE overestimates (1.28–1.48) and NODE underestimates (0.35). We removed the statement that a positive exponent confirms chaos and interpret λ₁ only for non-failed runs, together with the other metrics.

Task fMRI ROI Correlations

We agree and revised the caption and text: the reconstructed ROI–task correlations are systematically stronger than in the data, since the model generates a smooth, noise-free version of the ROI time series. We no longer report MMD for fMRI (general response, point 6).

Quantification of state alignment

We now report the ARI (1 = identical partitions, 0 = chance), which needs no mapping between states and regimes (Appendix C.1.6, Table 6). Since chaotic free runs diverge from any reference, we do not align them step by step, but compute: (1) the gate’s partition of the true test trajectory; (2) the model’s states along its own free run, with true regimes assigned by a classifier fitted on labelled data (99–100% held-out accuracy); and (3) the true attractor labelled by nearest free-run points (main measure). Only points near the true attractor are scored, with a threshold set from true data alone; we introduced this rule after observing that off-attractor points were being labelled, and apply it to all models and to the true system (0.98–1.00). We also report regime occupancy, since ARI can be high for free runs that barely leave one regime. Results are summarized in the general response (point 3). The K ablations now use 5 seeds with error bars.

Part 2: Presentation of existing DSR Literature

Autonomous generation

We agree with the point raised by the reviewer. Training reconstructs short windows from inferred initial states (less than half a Lyapunov time on Lorenz-63), which is closely related to multiple-shooting and sparse teacher forcing (Hess et al, 2023; Mikhaeil et al., 2022) and explains the failure at dt = 0.1 (Appendix A, revised). An ablation supports this (Appendix C.3, Table 4): shortening the training window from 50 to 20 steps improves rSNODE, and with a shorter encoder input λ₁ reaches 1.02 (true 1.01) and D_stsp drops from 5.1 to 1.9 at K = 4 (separately trained; main results use the original setting). We also agree that data-conditioned inference does not prevent autonomous generation, and removed the claims that our training is uniquely autonomous and that other models are not “truly generative”.

Reconstruction vs. generation

We apologize for mischaracterizing Koppe et al. (2019), who explicitly evaluate free-running behaviour with state-space and power-spectrum measures. The introduction and related work now state that free-running evaluation is standard practice in DSR and cite the works listed.

Our contribution

Switching state-space models such as rSLDS and SNLDS have mainly been trained and evaluated for posterior inference and segmentation rather than as long-term surrogates. Our contribution is to bring the DSR perspective to this model class: evaluating switching models by their free-running behaviour, testing whether nonlinear regime dynamics are needed (rdSLDS, rSNODE, rSLDS), and introducing the switching Lorenz benchmark for this purpose. We revised the abstract and introduction accordingly.

We thank the reviewer again for the thorough review, which has substantially improved the paper, and welcome any further comments.

Reviewer_8dqfOpenReview ↗

Reply to rebuttal

I appreciate the substantial revisions by the authors and believe they improve the paper. I also appreciate that the authors corrected how the existing DSR literature is presented and that they no longer claim their training to be more focused on autonomous generation than previous approaches, which resolves most of my second point. What remains here is that the text still states that PLRNNs "lack switching mechanisms altogether" (and the abstract now says that RNNs "lack discrete states"), which I'd not agree with, since PLRNNs are piecewise-linear and, although they share parameters between their linear regions, they switch between these linear regimes depending on the state (see e.g. Brenner et al., 2024).

The revisions also show some of the original claims did not hold, e.g. emphasizing autonomous rollouts as the key novelty. With the new comparison added, rSLDS outperforms rdSLDS on the Lorenz systems (as stated now by the authors). The original claim that only rSNODE reconstructs the switching Lorenz benchmark is also not cleanly visible in the new results. The new ablation over sequence lengths shows that the main results are run at a setting that is rather poor in performance.

Also, the newly added rSLDS baseline is not included in any of the significante tests, but e.g. the abstract states that rSNODE has the best spectral agreement and regime recovery among all models (which to me includes rSLDS as well).

To the new point of D_stsp favoring overly diffuse distributions: while the author's noise example is quite artificial/extreme, I agree. This is the reason to, as noted by the authors, always contextualize this metric together with e.g. D_H (which catches exactly such errors, as visible in the newly added example). However, I find that the authors only invoke this caveat where rSLDS is performing favorably. Similar are the remarks in the switching Lorenz results and the discussion that rSLDS with a leading Lyapunov exponent of 0.71 (true value 1.01) is "barely chaotic", while rSNODE's exponents are only said to be underestimated on Lorenz-63 (0.72-0.78, true value 0.906) and overestimated on switching Lorenz (1.28-1.48). I think that's not fairly portrayed since 0.71 is 30% below the true value which is the same size of error as for rSNODE in the other direction.

Taken together, the revisions were quite substantial and improved some of the issues. However, they were not only extensive in scope but also changed the main results and central claims. Besides the remaining time being too short to thoroughly assess all the revisions, I think this warrants a new manuscript anyways, with clearer framing/scope and more thorough evaluations. I therefore stay at my rating.

Reviewer_dX8hOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies dynamical systems that combine continuous dynamics with discrete switching. The authors propose a latent rollout framework where the encoder is only used to initialize the latent state, and the model then generates the rest of the trajectory autonomously. They introduce rdSLDS, which uses differentiable switching between linear dynamics, and rSNODE, which replaces the linear dynamics with mode-specific Neural ODEs. The models are tested on Lorenz, NASCAR, a new switching-Lorenz benchmark, and task-fMRI data.

Critical strengths

The paper gives a clean comparison between linear, nonlinear, and switching dynamics, and the synthetic benchmarks make the trade-offs easy to see. And the switching Lorenz benchmark is a reasonable small testbed.

Critical weaknesses

The difference relative to prior switching dynamical models is not very clear to me (see questions).

Seems the the statistics can't claim strong advantage over baselines and previous methods (see questions).

Questions for authors

could the authors clarify the main distinction from the SNLDS of Dong et al. (ICML 2020)? rSNODE seems mainly to replace these transitions with Neural ODE flows, and [2] does similar things.

Do you actually select one discrete mode, or do you keep using the soft Gumbel-Softmax mixture in Eq. 9? If it is still soft, does the switching-Lorenz result hold with hard one-hot mode selection?

In Discussion, “diverges on most seeds across all tested K”, seems doesn't matching your Fig. 4G, rdSLDS seems ok when K=2 and K=4.

In Fig. 4G, it seems only NODE vs rdSLDS K=2 (p = .001) clears your own Bonferroni-corrected threshold of α = 0.0083.

About the MMD setup, so a quick clarification: in Eq. (19), is each sample a single time point, a window, or a whole trajectory? If it is single time point, wouldn’t MMD be largely insensitive to temporal ordering.

[1] Dong, Zhe, et al. "Collapsed amortized variational inference for switching nonlinear dynamical systems." International Conference on Machine Learning. PMLR, 2020. [2] Poli, Michael, et al. "Neural hybrid automata: Learning dynamics with multiple modes and stochastic transitions." Advances in Neural Information Processing Systems 34 (2021): 9977-9989.

2 comments

We thank the reviewer for the positive assessment and the precise questions, several of which revealed inconsistencies we have now fixed. Please also see our general response.

Relation to SNLDS (Dong et al., 2020) and Neural Hybrid Automata (Poli et al., 2021)

We now discuss both in Sections 2.1 and 2.2 and removed statements implying that combining discrete regimes with nonlinear regime dynamics is new. SNLDS belongs to the same model family. It differs in that it fits an amortized posterior over the full latent trajectory and marginalizes discrete states via forward–backward, whereas our models infer only the initial state and roll out the learned dynamics over each training window (related to multiple shooting, so not claimed as novel); it is evaluated primarily for segmentation, whereas we evaluate long-term free-running behaviour; and it uses discrete-time transitions, whereas rSNODE uses continuous-time vector fields with switching on the latent state and previous regime. NHA also uses mode-conditioned neural ODEs, but targets stochastic hybrid systems with explicit event times and state jumps and is trained in stages; in its deterministic setting, modes are selected from the current state, similar to rSNODE. We agree these are design differences within one family whose value must be shown empirically; a direct comparison was not feasible within the rebuttal period. We added rSLDS as a switching baseline (general response, point 4).

Hard states during inference

Thank you; this was not clear in the paper. During training, both switching models use the soft Gumbel-Softmax mixture, with the temperature annealed from 1.0 to 0.1. During generation, the model returns a one-hot vector. All reported results, including switching Lorenz, therefore already use hard one-hot mode selection, and the plotted states are the model’s one-hot outputs without further processing. We added this description to Sections 3.1 - 3.3.

Agreement between Figure 4 and Discussion

Thank you; the statement was imprecise. On switching Lorenz, rdSLDS fails on 2, 9 and 11 of 15 seeds for K = 2, 4, 8, and its remaining runs have a D_stsp of 15.5, 11.9 and 7.6 (rSNODE about 5, GT 0.53). On chaotic Lorenz, it fails on 2 of 15 seeds at K = 2, but its free runs remain far from the attractor (D_stsp 6.5–10.4 vs. 0.6–0.8 for rSNODE). The Discussion now states these numbers; failures are reported for every model (Table 5).

Significance of model comparisons

Correct: for MMD, only NODE vs. rdSLDS (K = 2) cleared the corrected threshold. With D_stsp on held-out data, rSNODE is significantly better than rdSLDS at matched K on all Lorenz systems (switching Lorenz: K = 2 ***, K = 4 **), and rSNODE (K = 4) is significantly better than NODE on all benchmarks; on chaotic Lorenz, rSNODE (K = 2) and NODE do not differ significantly (Table 7). D_H and MMD boxplots with tests are in Figures 10, 13, 15 and 28.

MMD metric setup Each sample is a single time point, so MMD is insensitive to temporal ordering, as is D_stsp; D_H, which compares power spectra, captures temporal structure. We corrected the description of MMD (Appendix C.1.4).

We hope these comments clarify your questions.

Reviewer_dX8hOpenReview ↗

Reviewer followup

Thanks for the reply and my concern is been addressed. I decide to keep my score.

Comments on the paper

2 comments

We thank all reviewers for their careful and constructive feedback. We summarize the main changes here and address each reviewer’s points individually. All changes in the revised PDF are marked in blue.

1. Evaluation metrics

We agree with the concerns about MMD as our primary metric: its bandwidth was estimated from the pooled true and generated samples, it does not capture temporal ordering, and it did not reliably separate good from poor reconstructions. We now use the standard DSR measures, computed with the public implementation of the Durstewitz lab: the state-space divergence D_stsp for attractor geometry and the power-spectrum Hellinger distance D_H for temporal structure. The largest Lyapunov exponent λ₁ is now computed from the Jacobians of the learned dynamics. MMD is reported as a complementary measure, with a bandwidth fixed per reference and three bandwidths reported separately.

No model was retrained for the main results; only the evaluation changed. All metrics are computed on held-out data: free runs are initialized from 8 test trajectories not used in training, run for 100,000 steps after a 1,000-step transient, and compared to 10 independent 100,000-step reference trajectories. The true system evaluated with the same procedure gives a noise floor (the metric obtained from the ground-truth runs, “GT”). Failed runs (diverging or leaving the reference range) are counted and reported for every model (Appendix C.1, Table 5). We also report a limitation of D_stsp: computed as KL(true ‖ generated), it strongly penalizes unvisited parts of the attractor but only weakly penalizes generated mass outside it, and therefore favours diffuse free runs (Appendix C.1.1, Figure 9). We therefore interpret D_stsp together with D_H, λ₁ and the reconstructions.

2. Statistics and claims

We agree that our original claims were not supported by the statistics we reported. We now report the Bonferroni-corrected test results on the D_stsp metric, including a comparison between K-matched rdSLDS and rSNODE (Appendix C.2, Table 7). On all three Lorenz systems, rSNODE achieves significantly lower D_stsp than rdSLDS at matched K and fails on at most one seed, whereas rdSLDS fails on up to 11 of 15 seeds on switching Lorenz. rSNODE (K = 4) has significantly lower D_stsp than NODE on all benchmarks. On NASCAR, whose regimes are linear, the linear switching models perform best. No model reaches the ground-truth floor on switching Lorenz. We revised the abstract, figure captions and discussion accordingly.

3. Regime recovery

We now quantify unsupervised regime recovery with the adjusted Rand index (ARI) in three versions that require no step-by-step alignment of free runs with the ground truth, together with the regime occupancy of the free runs (Appendix C.1.6, Table 6). On switching Lorenz, rSNODE (K = 4) achieves the highest attractor ARI (0.37, vs. 0.21 for rSLDS and at most 0.16 for rdSLDS) and reproduces the true regime occupancy; recovery is nevertheless partial for all models, and we corrected our claim that rSNODE recovers “the correct two-state segmentation”. On NASCAR, a truly piecewise linear system, rSLDS and rdSLDS identify the regimes best. All generations use hard (one-hot) regime selection.

4. Baseline

We added rSLDS (Linderman et al., 2017; ssm implementation), evaluated with the same pipeline (Appendix C.2, D.5). On chaotic and noisy Lorenz, rSNODE has a lower median D_stsp than rSLDS (e.g. 0.58 vs. 2.82 and 0.52 vs. 1.91 at K = 4), and rSLDS fails on up to 13 of 15 seeds on chaotic Lorenz. On switching Lorenz, rSLDS attains a competitive D_stsp, but its free runs are diffuse, have a higher D_H (0.28–0.40 vs. 0.20–0.23), and are barely or not chaotic (λ₁ = −0.90 / 0.14 / 0.71 for K = 2 / 4 / 8; true value 1.01). On NASCAR, the linear switching models perform best (rdSLDS on D_stsp, rSLDS on D_H and regime recovery). Since rSLDS outperforms rdSLDS on the Lorenz systems, we present rdSLDS as the linear-regime variant of our framework rather than as an improvement over rSLDS.

5. Positioning relative to prior work

We corrected our description of Koppe et al. (2019) and now state that free-running evaluation is standard practice in DSR. Our short-window training is presented as related to (sparse) teacher forcing (Hess et al., 2023), supported by a window-length ablation (Appendix C.3, Table 4). We added a discussion of SNLDS (Dong et al., 2020) and Neural Hybrid Automata (Poli et al., 2021). Our contribution is now framed as evaluating switching state-space models as long-term DSR surrogates, testing whether nonlinear regimes are needed, and the novel switching Lorenz benchmark.

6. Task fMRI

We no longer report MMD for the fMRI data, given the issues discussed above. We report MSE and the power-spectrum correlation from the original evaluation (Appendix C.4, Table 8), since the benchmark protocol (independent references, long free runs) does not apply to short single-subject runs.

We believe these changes substantially improve the soundness of the paper, and we thank the reviewers for their contribution. We welcome any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.