acceptodds
Under review as a conference paper at ICLR 2027

SyLVER: symbolic equation recovery from neural population recordings

Abstract

Understanding neural computation requires population dynamics models that are predictive and interpretable. Sequential variational autoencoders (SVAEs) recover low-dimensional latent trajectories from high-dimensional recordings but their learned dynamics are challenging to interpret. Sparse regression methods such as SINDy yield interpretable closed-form equations but require the state to be directly observed. This introduces a central challenge when applying these methods when the underlying state is unobserved: a latent state must first be inferred by a separate model, and equations are then fit to those estimates which were inferred without any knowledge of the equation. To address this challenge we introduce Symbolic Latent Variable Equation Recovery (SyLVER), a sequential VAE whose transition drift is a sparse dictionary of candidate terms that provide a closed-form dynamics equation. Because inference is structured, the drift also drives the approximate posterior through the predict step of the filter, so the dynamics and the inference network share one latent coordinate system and are trained jointly using block optimization. The recovered equation is thus constrained by inference itself rather than regressed onto post-hoc state estimates. We evaluate on the Lorenz system, van der Pol oscillator, and the MC Maze reaching dataset, and demonstrate comparable latent inference, forecasting, and coefficient recovery against two established methods, XFADS and SINDy Autoencoder.

Primary area:
learning on time series and dynamical systems
Keywords:
equation discovery, sparse identification of nonlinear dynamics (SINDy), sequential variational autoencoder, latent dynamical systems, structured variational inference, state-space models, neural population dynamics, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_yj8QOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

They suggests a VAE for neural population recordings with a latent transition drift, named SyLVER, modelled with a sparse linear combination of polynomials. To do the inference part they used structured filtering with a per-bin MLP and a reverse-time GRU that produce Gaussian potentials. The posterior marginals for the inference come from a moment-matched predict step through the sparse drift. They evaluated the method on van der Pol, pendulum, and Lorenz system, and on the MC-Maze motor cortex dataset, where the authors report decodable velocity and probe the learned dynamics.

Critical strengths

Well-motivated problem, I agree with that the two-stage pipelines can inherit errors of the inference step. Placing the sparse drift inside the structured filter is a clear idea and can shapes the predictive distribution and receives gradient through the ELBO. This makes sense and clearly explained design. Also honest reporting of failure modes where in they mention that Lorenz recovery fails, this is informative even though, as noted below it undercuts the paper’s claims.

Critical weaknesses

1- Table 1 reports single runs with no error bars while the numbers are very close, and SyLVER does not beat SINDy-AE on forecasting. On MC-Maze, no standard neural latents benchmark metrics like co-bps/etc or competing methods are reported, even though the paper says the dataset was chosen for this kind of comparison and make sense to use those metrics too and report them, see table 1 of (Pei, 2021).

2- As presented here the main claims are not supported by the results. The abstract and discussion say SyLVER recovers equations but in the Lorenz experiment it misses most of the true terms (Table 2). The van der Pol and pendulum, recover the right coefficient and they use a decoder fixed to the true readout. That removes the core difficulty the paper is meant to solve, which is learning sparse dynamics through a learned observation model. The introduction and discussion claim sparse equations, but Section 4.2 and the table 2 says the learned equation is dense ( see van der Pol example) and clearly wrong for Lorenz (missing dynamics for the x), can you clarify what is happening here? as I understand here the key claim is contradicted by the results as as shown here.

3- Some important baselines are missing. The paper argues that joint learning beats a two-stage pipeline, but never compares against one of them (for example, XFADS latents followed by SINDy). It also does not compare against SVISE (you put this as one of the closest models to yours in the discussion but never compared against or being used as a baseline), which as I understand has the closest prior method. Also, none of the proposed components (block optimization or masked-bin training) is ablated.

Pei, F., Ye, J., Zoltowski, D.M., Wu, A., Chowdhury, R.H., Sohn, H., O'Doherty, J.E., Shenoy, K.V., Kaufman, M.T., Churchland, M.M., Jazayeri, M., Miller, L.E., Pillow, J.W., Park, I.M., Dyer, E.L., & Pandarinath, C. (2021). Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity. ArXiv, abs/2109.04463.

Questions for authors

1- The text reports 0.53 for ridge on spikes, while the Figure 3 caption reports 0.76. What does the latent model add If ridge results on raw spikes beats the MLP on latents (0.74)? can you clarify this

2- Table 3 says Lorenz uses Poisson observations, but Figure 2 and Table 1 say Gaussian. Which was used?

3- How SyLVER performs on van der Pol and pendulum with a learned decoder instead of the true readout if we use the same alignment as SINDy-AE does? I am asking this since without this the recovery results don’t test the method’s main claim.

4- How SyLVER compare with a two-stage baseline or with SVISE when the observation map is known?

5 comments
AuthorsOpenReview ↗

Response to Reviewer yj8Q (1/2)

We thank the reviewer for a careful and detailed review, and for recognizing the motivation for placing the sparse drift inside the structured filter. The reviewer's points on result consistency, matched baselines and the learned-decoder setting are well taken, and we will address each of them; we outline our plan below. Several points overlap with our responses to Reviewers k6du and msAG, and we refer to those responses for the full details where noted.

1. Equation recovery on Lorenz and van der Pol

We agree that the submitted results did not support the claims as presented, and that the reporting across Table 1, Table 2 and Figure 2 was inconsistent. We are consolidating these results. Since submission we have rerun the synthetic systems with the decoder fixed to the true readout, the same setting as the van der Pol and pendulum rows. In this setting SyLVER recovers all seven true Lorenz terms with the correct sign, in the rescaled generating frame:

TrueSyLVER
x˙\dot x−10x+10y-10x + 10y−17.58x+12.67y-17.58x + 12.67y + 4 terms
y˙\dot y28x−y−10xz28x - y - 10xz27.85x−1.18y−9.70xz27.85x - 1.18y - 9.70xz + 1 term
z˙\dot z−2.67z+10xy-2.67z + 10xy−2.75z+8.69xy-2.75z + 8.69xy + 3 terms

The y˙\dot y and z˙\dot z coefficients are close to their true values. The x˙\dot x coefficients have the correct support and sign but inflated magnitudes, and there are eight small terms outside the true support. The submitted Lorenz row instead came from a run with a learned decoder, in which the equation is expressed in a different latent frame and cannot be compared term by term with the physical equation (see our response to Reviewer k6du, §4). Table 2 therefore mixed two settings, which we should have made explicit.

On van der Pol, we would like to gently clarify that the recovered equation is not dense. The dense equation discussed in Section 4.2 refers to MC-Maze. For van der Pol, y˙=−0.97x+1.54y−1.48x2y\dot y = -0.97x + 1.54y - 1.48x^2y is recovered with no terms outside the true support. x˙\dot x recovers the true term (0.95y0.95y) together with five small additional terms (Table 4), which we will report with seed spreads.

We will revise the abstract, introduction and discussion so that each claim states the setting in which it holds.

2. Learned decoder and alignment (Q3)

We agree that the fixed-decoder setting removes part of the difficulty the paper targets. In the revision, every synthetic experiment will be reported both with the decoder fixed to the true readout and with a learned decoder. With a learned decoder, the latent is identifiable only up to an affine transformation, so the support and coefficients of the equation depend on the frame (see our response to Reviewer k6du, §4, for a worked example).

SINDy-AE learns a nonlinear decoder, and its equations are compared with the true ones after a post-hoc change of variables: a permutation of the latents together with a per-coordinate scaling and shift (Champion et al., 2019; Gao & Kutz, 2022). We will apply this same alignment to SyLVER's learned-decoder runs on van der Pol and pendulum, as the reviewer suggests. In addition, we will fit a full affine map from the learned to the true latents and transform the recovered coefficients exactly into the physical frame, which is possible because the polynomial dictionary is closed under affine maps. We will apply both procedures to all methods (see our response to Reviewer msAG, §1). We would welcome the reviewer's view on whether this comparison addresses their concern, or whether there is another test of the learned-decoder equations they would find more convincing.

3. Baselines and ablations (Q4)

We are adding the following, as described in our response to Reviewer msAG (§1, §2):

  • Two-stage pipeline. XFADS latents followed by sparse regression, with the same data, latent dimension, dictionary and computational budget as SyLVER.
  • SVISE (Course & Nair, 2023) in the setting where the observation map is known, which is the comparison the reviewer asks for in Q4.
  • SINDy-AE with matched dictionaries and decoder information.
  • Ablations of the horseshoe prior, the coefficient optimizer and masked-bin training.

(continued in next comment)

AuthorsOpenReview ↗

Response to Reviewer yj8Q (2/2)

4. Seed spreads and NLB metrics

All synthetic results, baselines and ablations will be reported over three seeds with mean ± standard deviation (see our response to Reviewer k6du, §3 and §5). Table 1 is intended as a parity baseline, and we agree that the spreads are needed to support even that. On MC-Maze, we will report the standard Neural Latents Benchmark metrics (co-bps, velocity R2R^2, PSTH R2R^2, forward prediction) against published results, following Table 1 of Pei et al. (2021).

5. Decoding numbers on MC-Maze (Q1)

We apologize for the inconsistency. The 0.53 and 0.76 values came from different decoding setups and were not reported consistently. We have since rerun MC-Maze with eight latent dimensions and a cubic dictionary, and evaluated decoding with the NLB protocol (ridge regression from inferred rates). Hand velocity decodes from inferred rates at R2=0.82R^2 = 0.82, and from the rates of the learned-equation rollout started just after movement onset at R2=0.44R^2 = 0.44. Setting Ξ=0\Xi = 0 drops the forecast to R2=−0.14R^2 = -0.14, so the forecast is carried by the learned equation. What the latent model adds over a decoder on the spikes is therefore a compact latent state with an explicit equation that forecasts the dynamics. We will report the NLB spike-smoothing baseline under the same protocol so that the comparison is direct (see our response to Reviewer msAG, §4, for the full updated results).

6. Observation model on Lorenz (Q2)

All synthetic results in Table 1, Table 2 and Figure 2 use Gaussian observations. Our data generator produces both Gaussian and Poisson variants of each system from the same latent trajectories, and the Poisson entry for Lorenz in Table 3 refers to that variant; it is an error in the table. We will correct it and state the observation model in every table and figure caption. We use Gaussian observations for the synthetic benchmarks because this is the setting in which the baselines are typically evaluated, and Poisson observations for spike counts.


To summarize, we will: consolidate the results so that every table and figure states its setting consistently; report the updated fixed-decoder Lorenz results; report all synthetic experiments with fixed and learned decoders, aligned as for SINDy-AE and through an exact affine transformation; add the two-stage, SVISE and matched SINDy-AE baselines and the three component ablations over three seeds; report NLB metrics on MC-Maze; and revise the claims to match the evidence. We are confident that this plan addresses the reviewer's concerns about soundness. We would welcome the reviewer's view on whether it does, whether there are other comparisons they would find convincing, and whether we should prioritize anything differently.

References

Champion, K., Lusch, B., Kutz, J. N., & Brunton, S. L. (2019). Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy of Sciences, 116(45), 22445–22451.

Course, K., & Nair, P. B. (2023). State estimation of a physical system with unknown governing equations. Nature, 622(7982), 261–267.

Gao, L. M., & Kutz, J. N. (2022). Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants. arXiv preprint arXiv:2211.10575.

Pei, F., Ye, J., Zoltowski, D., Wu, A., Chowdhury, R. H., Sohn, H., O'Doherty, J. E., Shenoy, K. V., Kaufman, M. T., Churchland, M., et al. (2021). Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity. arXiv preprint arXiv:2109.04463.

Thank you for the detailed rebuttal. Most of the evidence I requested (matched baselines, learned-decoder recovery, ablations, seed spreads and NLB metrics) is promised but not yet provided also my remaining concerns are unanswered:

  1. The rebuttal concedes that no sparse equation will be claimed on neural data, and my question on decoding from latents versus raw spikes is still unanswered.

  2. Clarifications here dont match up as for example which van der Pol equation has the spurious terms and the Lorenz observation model. The new Lorenz result still relies on an oracle decoder which has more spurious than true terms and comes from a single run.

The new comment didn't clarify the main claim and the planned revisions with promised experiment which I believed needs a fresh review. I therefore maintain my score.

Reviewer_k6duOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The goal of this paper is to develop a latent-state equation discovery model. The approach the authors take builds off of SINDy and design a more RNN-like autoencoder framework to contain the SINDy-style equation discovery in the latent states. Specifically the authors define a forward and reverse recursion based on multi-layer perceptrons with the recurrent-equation version of SINDy as a driving force. The coefficients of this latent equation are then the coefficients of the differential equation to be discovered.

The idea is interesting, however not clear just how different the proposal is from the (compared to) SINDy-AE, or SINDy in an autoencoder that was previously explored in the literature. This concern is especially evident in the comparisons where it is not clear that the proposed method has any serious advantages. Additional seeds/diversity over example traces etc. to get a spread of values and more exploration of why their method misses critical terms in the discovered equations would go a long way towards solidifying this. Without confidence on simple equations it is difficult to really read into the real data example.

As an aside, there's a lot of very off phrasing. Things like hanging segments separated by a comma or awkward passive voice. The authors might do well to do a careful read-through for these issues since it distracts form the content of the work.

AI statement: I did not use AI for my review at all.

Critical strengths

The idea of using the recurrent SINDy model inside the AE state is interesting and might be beneficial in some cases.

Critical weaknesses
  • There is a bit of tension between describing things in a "proper" probabilistic setting and the use of more modern ML lingo. As one example, Equation 10 is a bit odd in it's description. The authors present two possible readouts. Is the idea that one is selected manually based on the application or is the implication that the selection is automated? This might be redundant with the discussion of the loss selection to some extent.
  • In a similar sense, the loss seems to be presented after a good bit of the inference sections.
  • I don't see any spreads of the values e.g., in Table 1, which means that given how close the values are it's hard to tell if there's any real benefit
  • Similarly, Table 2 doesn't show the method clearly outperforming SINDy-AE. In fact, the presented method is the only one that seems to miss vital terms, indicating that it's completely missing the actual equation.
  • The above issues make it very hard to read into the real data example with any confidence
Questions for authors
  • Why does the methods miss vital terms, specifically in the lorenz case.
  • What is the spread over, e.g., random seeds and different sample trajectories for the different methods?
6 comments
AuthorsOpenReview ↗

Response to Reviewer k6du (1/3)

We thank the reviewer for their careful reading and concrete suggestions. The requests for seed spreads and for an explanation of the Lorenz results point to real gaps in how we presented the results. We address both below. We first clarify how SyLVER differs from SINDy-AE, since the current draft does not make this prominent enough.

1. SyLVER is a sequential variational autoencoder (VAE), not an autoencoder with SINDy dynamics

In SINDy-AE (Champion et al., 2019), the latent ztz_t is a deterministic function of the input at each time point, and the equation enters as a regularizer matching z˙\dot z to Θ(z)Ξ\Theta(z)\Xi. SyLVER is instead a probabilistic state-space model (Eqs. 8–10). The latent evolves stochastically with learned process noise QQ, and inference returns an approximate posterior over the entire trajectory z1:Tz_{1:T} given y1:Ty_{1:T}, rather than a compressed code per time point. None of these features are present in SINDy-AE.

The learned equation (Eq. 6, 9) is part of how that posterior is constructed. Each marginal πt\pi_t comes from a predict step through the drift followed by an update with the data potential (Sec. 3.2). Eq. 12 writes the posterior mean as a precision-weighted average of the drift's prediction and the observation. This has three consequences:

  • Temporal structure. The posterior uses past observations through the predictive πˉt\bar\pi_t and future observations through the reverse-time potential. Noise in the dynamics (QQ) is therefore modeled separately from noise in the measurements (the observation model). SINDy-AE does not have dynamics noise because it is a deterministic ODE.
  • The equation shapes inference. Ξ\Xi receives gradient both through the reconstruction term, via the filter, and through the dynamics KL (Eqs. 13–14). It is not fitted to fixed state estimates.
  • The likelihood is modular. The same construction takes a Poisson likelihood for spike counts, which a deterministic reconstruction objective does not naturally accommodate.

To make this distinction clear, we will add the following comparison table to Section 3:

SINDy-AESyLVER
LatentDeterministic, per time pointPosterior over z1:Tz_{1:T}
Process noiseNoneLearned QQ
Role of the equationDerivative regularizerDefines the predictive πˉt\bar\pi_t inside the posterior
Observation modelReconstruction lossGaussian or Poisson likelihood

2. The readout in Eq. 10, and the placement of the loss

The readout (Eq. 10) is chosen by data modality, not automatically. We will change our notation for Eq. 10 to avoid this confusion. For spike counts (MC-Maze) we use Poisson. For the synthetic systems we use Gaussian observations. SINDy and its noise-robust variants evaluate on synthetic systems with Gaussian measurement noise and a known observation map (Brunton et al., 2016; Messenger & Bortz, 2021; Fasel et al., 2022). SVISE assumes a given observation function with additive noise (Course & Nair, 2023b). Gaussian observations are also the more informative regime for verifying equation recovery before moving to spikes. We will state this explicitly and correct an inconsistency in Table 3. We will also move the objective ahead of the inference details so the reader knows what is being optimized before the recursion is described.

3. The purpose of Table 1

Table 1 is intended as a parity baseline, not a claim of superiority. It shows that SyLVER matches dedicated latent variable models on latent recovery and forecasting while also returning a closed-form equation with a posterior over its coefficients. In other words, SyLVER does not perform worse than standard approaches at the cost of interpretability. We will include text about this point in Table 1's caption so this is more clear. We agree that, because the values are close, spreads are essential to support a parity claim (see §5).

(continued in next comment)

AuthorsOpenReview ↗

Response to Reviewer k6du (2/3)

4. Why terms are missed on Lorenz

There are two reasons.

(a) Coordinate frame. With a learned decoder, the latent is identifiable only up to an affine transformation. The dynamics are unchanged by such a transformation, but the support and coefficients of the equation are not. A minimal example is the sparse system

x˙1=−x1,x˙2=−2x2+x12\dot x_1 = -x_1, \qquad \dot x_2 = -2x_2 + x_1^2

which has three terms. Under a 45∘45^\circ rotation, u=(x1+x2)/2u = (x_1+x_2)/\sqrt{2} and v=(x2−x1)/2v = (x_2-x_1)/\sqrt{2}, the same system becomes

u˙=−1.5u−0.5v+0.354u2−0.707uv+0.354v2\dot u = -1.5u - 0.5v + 0.354u^2 - 0.707uv + 0.354v^2 v˙=−0.5u−1.5v+0.354u2−0.707uv+0.354v2\dot v = -0.5u - 1.5v + 0.354u^2 - 0.707uv + 0.354v^2

which has ten terms. The trajectories and predictions are identical, but the equation is dense. In the submitted Lorenz run, the latent frame was learned, so the recovered equation is the true one expressed in a rotated and shifted frame. Comparing it term by term to the physical equation therefore conflates a change of coordinates with an error in the dynamics. The van der Pol and pendulum rows used a decoder fixed to the true readout, so Table 2 mixed the two settings. We should have made this explicit and our revised manuscript will make this more clear.

(b) Collinearity on the attractor. In the submitted run, the xyxy term in z˙\dot z was largely within the span of the diagonal quadratic terms over the region the attractor visits. Its weight was therefore absorbed by a surrogate.

Two settings, reported side by side. In the revision, every synthetic experiment will be reported under both settings, for all methods:

  1. Anchored frame (decoder fixed to the true readout). This is the only setting in which recovered coefficients can be compared term by term with the generating equation. We use it for exact coefficient comparison, and SINDy-AE is run under the same decoder information and the same dictionary.
  2. Learned decoder. This is the setting the method targets. Here we report latent recovery and forecasting accuracy, and the recovered equations mapped through the fitted affine transformation into the physical frame.

The anchored frame is a controlled test of coefficient recovery. It complements the learned-decoder results; it does not replace them.

Updated Lorenz results (anchored frame). With the decoder fixed to the true readout, SyLVER recovers all seven true terms with the correct sign. In the rescaled generating frame:

TrueSyLVER
x˙\dot x−10x+10y-10x + 10y−17.58x+12.67y-17.58x + 12.67y + 4 terms
y˙\dot y28x−y−10xz28x - y - 10xz27.85x−1.18y−9.70xz27.85x - 1.18y - 9.70xz + 1 term
z˙\dot z−2.67z+10xy-2.67z + 10xy−2.75z+8.69xy-2.75z + 8.69xy + 3 terms

The y˙\dot y and z˙\dot z coefficients are close to their true values. The x˙\dot x coefficients have the correct support and sign but inflated magnitudes, and there are eight small terms outside the true support. Forecast R2R^2 improves from 83.8 to 87.7, with latent R2R^2 of 99.8. In the learned-decoder setting, accurate long-horizon prediction indicates that the true dynamics are being learned even when the raw equation is dense. Would the reviewer find the learned-decoder comparison, with equations mapped to the physical frame, a useful addition?

(continued in next comment)

AuthorsOpenReview ↗

Response to Reviewer k6du (3/3)

5. Spread over seeds and trajectories

We will report results over three random seeds and multiple sample trajectories for all methods. Tables 1 and 2 will show mean ± standard deviation, and Table 2 will also show support recovery frequency. Three seeds is what our compute allows within the discussion period. If time permits we will add more, and the camera-ready version will include them.

We expect performance on van der Pol and pendulum to be saturated, with small seed-to-seed variation across methods. Forecasting on Lorenz should vary more across seeds for every method, because chaotic divergence amplifies small differences in the learned drift and the initial state. We will report this variation.

6. Writing

We thank the reviewer for flagging the phrasing. We are doing a full pass for hanging clauses and passive constructions, and making the results description consistent across sections, particularly for MC-Maze.


To summarize, we will: clarify the model and add the comparison to SINDy-AE; restructure the methods so the objective comes first; report all synthetic comparisons under both anchored and learned-decoder settings, with matched decoder information and dictionaries for all methods; include the updated Lorenz results; and add seed spreads throughout. We are confident that these changes address the concerns about soundness and novelty. We would welcome the reviewer's view on whether this plan resolves their concerns, whether the learned-decoder comparison would strengthen the case, and whether there are other experiments they would consider necessary. If these revisions resolve the reviewer's concerns, we would welcome a re-evaluation of the score and we are available to consider any remaining concerns.

References

Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2016). Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15), 3932–3937.

Champion, K., Lusch, B., Kutz, J. N., & Brunton, S. L. (2019). Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy of Sciences, 116(45), 22445–22451.

Course, K., & Nair, P. B. (2023). State estimation of a physical system with unknown governing equations. Nature, 622(7982), 261–267.

Fasel, U., Kutz, J. N., Brunton, B. W., & Brunton, S. L. (2022). Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control. Proceedings of the Royal Society A, 478(2260), 20210904.

Messenger, D. A., & Bortz, D. M. (2021). Weak SINDy: Galerkin-based data-driven model selection. Multiscale Modeling & Simulation, 19(3), 1474–1497.

Reviewer_msAGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a SyLVER, an interpretable dynamical equations from high-dimensional neural population recordings while jointly inferring latent states. It replaces an end-to-end transition drift with a dictionary expansion, under a horseshoe sparsity prior. Unlike post-hoc equation fitting, the proposed symbolic drift participates directly in the prediction step of a structured variational inference recursion. The SyLVER also supports Poisson and Gaussian observation models. The results demonstrate competitive latent recovery and forecasting, with partial equation recovery on anchored synthetic systems. I would be willing to raise my score if the authors address the following concerns with convincing evidence, particularly matched baseline comparisons, component ablations, and validation of sparse recovery on neural data.

Critical strengths
  • Recovering interpretable dynamics from noisy, partially observed neural activity is a meaningful goal beyond accurate latent reconstruction.

  • Coherent integration of inference and dynamics. Embedding the symbolic drift inside the posterior recursion provides a direct mechanism through which equation learning can influence state estimation.

  • The model explicitly represents latent uncertainty and process noise and supports a Poisson observation model appropriate for spike counts.

  • The Lorenz results and MC-Maze pruning experiment expose the practical importance of coordinate choice and the trade-off between sparsity and predictive accuracy.

Critical weaknesses
  • Baseline coverage is too limited to establish the central contribution. The main comparisons are restricted to XFADS and SINDy-AE. These are relevant, but neither directly tests whether embedding symbolic drift in posterior inference improves over a matched two-stage pipeline. Comparisons with XFADS followed by sparse equation fitting, appropriately configured probabilistic equation-discovery methods, and standard latent models on MC-Maze are needed. The existing SINDy-AE comparison should also match decoder information and dictionaries.

  • SINDy-AE already jointly learns coordinates and equations, while XFADS provides structured probabilistic inference. Comparisons against these models do not isolate the benefit of inserting symbolic drift into the posterior recursion. A matched XFADS-plus-equation-fitting pipeline, together with controls for the horseshoe prior, masked training, and a specialized coefficient optimizer, is needed.

  • The reverse-time GRU incorporates future observations. Although Figure 2 refers to causal initial conditions, the procedure for recomputing recognition potentials at each forecast cut is insufficiently explicit. Masking combined potentials after the GRU also does not ensure that earlier posterior states exclude information from the masked interval. These are distinct issues requiring explicit protocols rather than an assumption that masking alone enforces prediction.

  • The successful van der Pol and pendulum SyLVER equations use the true decoder to anchor physical coordinates. SINDy-AE equations are reported in learned coordinates, with different dictionary choices in the pendulum experiment. Because support and coefficient values depend on coordinates, these comparisons do not establish superior recovery under an unknown observation map. High latent R² after affine alignment does not resolve this issue.

Questions for authors
  • Can a compact MC-Maze equation preserve the dense model’s forecasting performance? Please provide a complexity–accuracy curve and the corresponding recovered equations.

  • Can SyLVER and SINDy-AE be compared under matched decoder information, dictionaries, and coordinate transformations, separating oracle-anchored and learned-decoder settings?

  • Does joint posterior coupling outperform a matched two-stage pipeline when data, latent dimension, dictionary, and computational budget are fixed?

  • What are the separate contributions of the horseshoe prior, masking strategy, and coefficient optimizer? Are these effects stable across seeds?

6 comments
AuthorsOpenReview ↗

Response to Reviewer msAG (1/3)

We thank the reviewer for a thorough and constructive review, and for stating clearly what evidence would change their assessment. The requests for matched baselines, component ablations and an explicit forecasting setup are well taken, and we will address each of them; we outline our plan below. We also thank the reviewer for recognizing the strengths of the work: placing the drift inside the posterior recursion and modeling process noise and spike counts explicitly. Several points overlap with our response to Reviewer k6du (the purpose of Table 1, coordinate frames, and seed spreads). We summarize those briefly here and refer to that response for the full explanation.

1. Matched baselines

We agree that the current comparisons do not isolate the benefit of placing the symbolic drift inside the posterior recursion. We are adding the following comparisons:

  • Two-stage pipeline. XFADS with an MLP drift, followed by sparse regression on its smoothed latents, with the same data, latent dimension, dictionary and computational budget as SyLVER. This will directly test whether joint posterior coupling improves over fitting equations post hoc (the reviewer's Q3).
  • Probabilistic equation discovery. SVISE (Course & Nair, 2023), which learns a sparse dictionary drift under a horseshoe prior with a Gaussian process state posterior.
  • SINDy-AE under matched information. The same dictionary as SyLVER for every system, including the pendulum. In the fixed-decoder setting, SINDy-AE's decoder is replaced by the true readout.
  • Standard latent models on MC-Maze. The Neural Latents Benchmark metrics (co-bps, velocity R2R^2, PSTH R2R^2, forward prediction), which we will report against published results (Pei et al., 2021).

Fixed vs. learned decoder. Every synthetic experiment will be reported in two settings, separating oracle-anchored and learned-decoder results as the reviewer asks in Q2. With the decoder fixed to the true readout, recovered coefficients can be compared term by term with the generating equation. With a learned decoder, the latent is identifiable only up to an affine transformation (Muratore & Mathis, 2026): the dynamics are unchanged, but the support and coefficients of the equation are not, so a sparse equation can appear dense in the learned frame. The transformed equation stays within the dictionary only for a polynomial dictionary; trigonometric terms are not closed under such maps (App. A.1). We give a worked two-dimensional example in our response to Reviewer k6du (§4). In the learned-decoder setting, we will report latent recovery and forecasting accuracy. We will also fit an affine map from the learned latents to the true latents, the same map used for the aligned R2R^2, and transform the recovered coefficients exactly into the physical frame. We will then report support recovery and coefficient error there, applying the same procedure to all methods. This follows the convention of comparing SINDy-AE equations after a change of variables (Champion et al., 2019; Gao & Kutz, 2022). SVISE requires a known observation map, so it appears in the fixed-decoder setting only. We would welcome the reviewer's suggestions for further tests of the learned-decoder equations, since this coordinate dependence is inherent to any method that learns its own latent frame.

(continued in next comment)

AuthorsOpenReview ↗

Response to Reviewer msAG (2/3)

2. Component ablations

We will ablate the three methodological components on the synthetic systems in the fixed-decoder setting, where the true equation is available for comparison. Our expectations, which the experiments will test, are:

  • Horseshoe prior (replaced by a Gaussian prior on Ξ\Xi). We expect substantially more spurious terms and a denser recovered support, since the Gaussian prior shrinks all coefficients uniformly rather than concentrating mass at zero while leaving large coefficients essentially unpenalized (App. A.1). Metrics: support precision and recall against the true equation, number of spurious terms, and coefficient error, together with forecast R2R^2 to confirm that the denser equation does not predict better.
  • Coefficient optimizer (preconditioned Newton step replaced by Adam). Adam's per-step change is approximately the learning rate in magnitude, independent of the gradient scale. With the learning rate of 10−310^{-3} used for the inference block (App. A.4), a coefficient of 28, as for the xx term in the Lorenz y˙\dot y, needs at least 2.8×1042.8 \times 10^4 steps to reach its value, while the coefficient 2.67 in z˙\dot z needs a tenth of that. Column standardization rescales the dictionary columns but does not equalize the true coefficients. We therefore expect much slower convergence for large coefficients under Adam. Metrics: in the interest of time, we will run both optimizers for an equal number of epochs and compare how far the coefficients move toward their true values, reporting coefficient error over training together with final coefficient error and forecast R2R^2.
  • Masked-bin training (removed). Without masking, the data potential can explain every bin directly, so the posterior is dominated by the encoder. The dynamics' share of posterior precision in Eq. 12 shrinks, and the dynamics KL carries little information about the drift. We expect the prior to become nearly uninformative, Ξ\Xi to be poorly determined, and forecasting to degrade. Metrics: forecast R2R^2 as a function of horizon, the dynamics' share of posterior precision Jˉt/(Jˉt+J~t)\bar J_t/(\bar J_t + \tilde J_t) from Eq. 12, and coefficient error and support recovery.

All ablations will be run over three seeds, as described in our response to Reviewer k6du (§5).

3. Causal forecasting and masking

Forecast initial conditions. All forecasts reported in the paper use a causal initial condition. At the forecast cut, the reverse-time potential is set to zero (htβ=0h^\beta_t = 0, with zero precision), so the state at the cut uses only observations up to and including that bin, through the filtered past and the local potential. The potentials are recomputed in this way at each cut. We will state this explicitly in Sec. 3 and make the figure captions consistent.

Masking during training. The reviewer is right that masking the combined potential after the reverse recursion does not remove information about the masked interval from earlier states: the reverse-time potential at bins before a gap summarizes observations inside it. Masking is a training device that forces the drift to carry the state across gaps; it is not used at evaluation, where the causal setup above applies. If time permits, we will add a variant that also masks the local potentials inside the gap before the reverse recursion, so that earlier states exclude the masked observations, and report its effect on equation recovery and forecasting. We will describe the two issues separately in the revision.

4. Compact equations on MC-Maze

Updated results. Since submission we have rerun MC-Maze with eight latent dimensions and a cubic dictionary. Following the NLB protocol, ridge regression decodes hand velocity from inferred rates at R2=0.82R^2 = 0.82, and from the rates of the learned-equation rollout started just after movement onset at R2=0.44R^2 = 0.44. For reference, XFADS with 40 latent dimensions reports reach-velocity decoding R2R^2 of 0.885 when smoothing and 0.527 in a 500 ms prediction window after a 200 ms context window, and LFADS reports 0.911 and 0.232 (Dowling et al., 2024, App. E.7). The forecast windows differ from ours, so we will rerun XFADS under our protocol for a direct comparison. From predicted latents, a nonlinear readout decodes velocity at R2=0.77R^2 = 0.77, compared with 0.160.16 for a linear readout. Setting Ξ=0\Xi = 0 drops the forecast to R2=−0.14R^2 = -0.14, and the rolled-out latents explain 74% of the variance of the inferred latents, so the forecast is carried by the equation rather than the initial state. Freezing each latent's dynamics at the cut identifies z3z_3 and z6z_6 as most critical for forecasting (velocity R2R^2 drops of 0.55 and 0.52). We will update the corresponding figure in the paper with these results and add further details to the appendix.

(continued in next comment)

AuthorsOpenReview ↗

Response to Reviewer msAG (3/3)

4. Compact equations on MC-Maze (continued)

Complexity–accuracy curve. We will report forecasting accuracy as a function of the number of retained terms, with the corresponding equations, for latent dimensions 6, 8, 10 and 12. Training at larger latent dimension is time-limited, so the full NLB metrics will be reported as the remaining runs complete.

A caveat on this curve. The dictionary size grows combinatorially with latent dimension: K=(d+pp)K = \binom{d+p}{p} gives 28 terms for d=6,p=2d = 6, p = 2, but 165, 286 and 455 for d=8,10,12d = 8, 10, 12 at p=3p = 3. There is a trade-off between latent dimensionality and the complexity of the function basis. A richer basis can describe the dynamics in fewer dimensions, whereas a higher latent dimension with a polynomial basis needs many more terms and longer training. Higher dimension therefore may not yield a sparser equation. This is a limitation of the polynomial dictionary rather than of the framework, and richer basis functions can relax it. Relatedly, the residual of a sparse fit can guide which terms to add to the dictionary, which we will discuss as a direction for future work.

Sparse recovery on neural data. MC-Maze has no ground-truth equation, and the density of the learned equation depends on the latent frame, as on Lorenz. For this reason we will interpret the neural-data equation through its predictive behavior and targeted analyses (per-latent ablation, the distilled leading terms, and the complexity–accuracy curve), rather than claiming a uniquely sparse equation.

5. Reporting

We acknowledge that results are currently reported inconsistently across the paper. We are consolidating them so that every table and figure states the observation model, decoder setting, forecast initial condition and number of seeds. We would welcome any further suggestions on presentation.


To summarize, we will: add the two-stage, SVISE and matched SINDy-AE baselines, together with NLB metrics on MC-Maze; report all synthetic experiments in fixed- and learned-decoder settings; ablate the horseshoe prior, the coefficient optimizer and masked training over three seeds; make the causal forecasting setup and the role of masking explicit; and report complexity–accuracy curves on MC-Maze across latent dimensions. We are confident that this plan addresses the reviewer's central concerns. We would welcome the reviewer's view on whether it does, whether there are other tests they would find convincing, particularly for learned-decoder equations, and whether we should prioritize anything differently. Once we have completed these revisions, we would be grateful if the reviewer would revisit their score in light of the strengths they noted and the importance of the problem.

References

Champion, K., Lusch, B., Kutz, J. N., & Brunton, S. L. (2019). Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy of Sciences, 116(45), 22445–22451.

Course, K., & Nair, P. B. (2023). State estimation of a physical system with unknown governing equations. Nature, 622(7982), 261–267.

Dowling, M., Zhao, Y., & Park, I. M. (2024). eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling. Advances in Neural Information Processing Systems, 37, 13458–13488.

Gao, L. M., & Kutz, J. N. (2022). Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants. arXiv preprint arXiv:2211.10575.

Muratore, P., & Mathis, M. W. (2026). Extracting governing equations from latent dynamics via multi-view contrastive learning. arXiv preprint arXiv:2606.13260.

Pei, F., Ye, J., Zoltowski, D., Wu, A., Chowdhury, R. H., Sohn, H., O'Doherty, J. E., Shenoy, K. V., Kaufman, M. T., Churchland, M., et al. (2021). Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity. arXiv preprint arXiv:2109.04463.

Comments on the paper

6 comments
AuthorsOpenReview ↗

General comment: results update for all reviewers (1/6)

We thank all reviewers for their continued engagement. Below we report the results that several reviewers asked for. All numbers are from a single seed unless stated; seed repeats were not possible within the time and compute available during the discussion period.

1. MC-Maze: NLB metrics

Metrics follow the NLB definitions and are verified against nlb_tools. We report results on a test set of held-out trials. For forward prediction, every model sees held-in spikes up to bin 34 and is rolled forward 10 bins by its own dynamics. SyLVER and SINDy-AE are scored at the posterior-mean rate, with a noise-free rollout for forward prediction. For XFADS we report the same, and in parentheses the forward-prediction bits per spike (fp-bps) from its own evaluation, which averages rates over sampled trajectories with process noise.

methodlatent dddynamicsco-bpsfp-bpsvel R2R^2PSTH R2R^2
Smoothing (NLB baseline)––.208–.601–
XFADS40GRU.276+.109 (+.083).847.532
XFADS8GRU.177+.014 (−.035).713.396
SINDy-AE on spikes (Poisson readout)8cubic polynomial, 9 terms.054−.852.163−.639
SyLVER, Ξ=0\Xi = 0 (control)8none.133−.047.650.211
SyLVER6cubic polynomial.160+.091.652.314
SyLVER8cubic polynomial.196+.122.800.388
SyLVER10cubic polynomial.211+.142.835.414
  • At matched dimension (d=8d = 8), SyLVER's equation outperforms XFADS on co-bps, forward prediction and velocity decoding; XFADS is better on PSTH R2R^2.
  • SyLVER at d=10d = 10 has the highest forward prediction of all models, including XFADS with 40 latent dimensions. XFADS at d=40d = 40 remains better on co-bps, velocity and PSTH.
  • SINDy-AE trained directly on the spikes fails to model these data. We use Champion et al.'s published training schedule (thresholding at 0.1 every 500 epochs, 5000 epochs plus 1000 of refinement) with two adaptations that spike data require. The reconstruction loss is replaced by a Poisson likelihood with an exponential readout, and, as NLB co-smoothing requires, the encoder sees only the 137 held-in neurons while the decoder predicts all 182, including the 45 held-out ones. As in the original method, the encoder maps each time bin to a latent state on its own, and the sparse equation is fitted through the chain rule to time derivatives of smoothed log firing rates. Thresholding reduces the equation from 862 active terms to 9; its forecasts stay bounded but are poor (fp-bps −.852). We attribute this to the per-bin encoder: without a filter that pools information over time, it cannot recover a usable state from sparse, noisy spike counts. No published SINDy-AE results on MC-Maze exist for comparison.
  • The learned dynamics carry information: against the Ξ=0\Xi = 0 control, the equation adds +.063 co-bps and +.169 fp-bps at d=8d = 8, +.063 and +.194 at d=10d = 10, and +.063 and +.142 at d=6d = 6. The control is trained identically but with the equation fixed at zero, so the latent state can only persist from bin to bin. The gap therefore measures what the learned equation itself contributes, beyond the encoder and readout: it improves both inference (co-bps) and forecasting (fp-bps), so SyLVER's performance is not coming from the encoder alone.
  • All models are below sequential-autoencoder state of the art (co-bps ≈ .36, velocity ≈ .89 on the leaderboard). We do not claim SyLVER is a state-of-the-art rate model; the claim is an explicit equation that forecasts as well as or better than a neural state-space model of up to four times its latent dimensionality (fp-bps +.142 at d=10d = 10 against +.109 for XFADS at d=40d = 40). Beyond placing SyLVER among methods with a similar analytical goal, it is important to state what it can do that they cannot: learn an explicit, interpretable equation directly from spike counts, inside posterior inference, at a latent dimension small enough to read, and forecast neural activity with it as well as or better than a neural state-space model of the same size. SVISE (Course & Nair, 2023) is not included on MC-Maze because it requires the observation model, the readout from latent state to observations, to be known in advance. For spike recordings this readout is unknown and must be learned, so SVISE cannot be applied; we compare against it on the synthetic systems, where the true readout is available (§4).
AuthorsOpenReview ↗

General comment: results update for all reviewers (2/6)

2. Two-stage pipeline comparisons on MC-Maze: sparse regression after a neural SSM

We fit SINDy to the drift of XFADS (sequentially thresholded least squares, choosing the sparsest model whose validation drift R2R^2 is within .02 of the dense fit), and separately compressed XFADS (d=40d = 40) to 8 dimensions with SINDy-AE.

pipelineterms keptfp-bps
XFADS d=8d = 8 → SINDy (cubic)655 / 1320 (50%)+.019
XFADS d=40d = 40 → SINDy (quadratic)12,146 / 34,440 (35%)–
XFADS d=40d = 40 → SINDy-AE (d=8d = 8, cubic)1041 / 1320 (79%)+.101
SyLVER d=10d = 10 (single stage)2860 (dense)+.142

Attempting a two-stage pipeline as requested exposed a fundamental limitation which ultimately demonstrates why SyLVER is an advance beyond existing approaches. Neural SSMs need many latent dimensions to be competitive: XFADS forecasts barely above zero at d=8d = 8 (fp-bps +.014) and reaches +.109 only at d=40d = 40. At d=8d = 8, sparse regression on XFADS keeps half of the 1320 terms; its rollouts stay bounded without noise, but 1% of trials diverge once process noise is added. Sparse regression at d=40d = 40 keeps 12,146 terms even at the sparsest acceptable threshold, and half of its rollouts diverge even without process noise. In short, a two-stage approach can be intractable as per our tests, demonstrating why the models need to be trained in tandem, exactly as SyLVER proposes.

Compressing the d=40d = 40 model to 8 dimensions with SINDy-AE adds a second bottleneck and reintroduces the same problem: the 8-dimensional equation is again dense (79% of terms) and forecasts less well than SyLVER (+.101 against +.142). SyLVER avoids both because the equation is learned at the working dimension, inside the filter that generates the states it is evaluated on, including across masked gaps where the state is the equation's own prediction, so it is trained on the states its rollouts actually visit. We have added the details in the appendix of the updated manuscript.

Both comparison methods also rely on repeated fits to reach a sparse equation: SINDy on a sweep over its sparsity threshold, and SINDy-AE on multiple restarts, from which Champion et al. (2019) report the sparsest model (on Lorenz, 2 of their 10 restarts found the 7-term equation). We applied both procedures (a threshold sweep for SINDy and 10 restarts for SINDy-AE), and on MC-Maze neither produced a sparse equation. They face the same identifiability problem as SyLVER: on low-dimensional synthetic systems it is mild, but on high-dimensional, noisy neural recordings refitting does not recover a sparse frame. This demonstrates that SyLVER is not lacking in a property that other methods possess – all sparse equation recovery methods struggle on this task.

AuthorsOpenReview ↗

General comment: results update for all reviewers (3/6)

3. What we claim on neural data

SyLVER's equation on MC-Maze is dense. Our post-hoc pruning, which selects terms for each latent dimension by one-step prediction, reduces it to 36–39 terms. The pruned equation, however, is worse on every metric and diverges at d=10d = 10, so we report the dense equation. At its default global scale, the horseshoe prior does not outweigh the likelihood on this dataset: the learned global scale is about 1.5, and the prior contributes a median of under 1% of each coefficient's posterior precision. Replacing it with a Gaussian prior therefore changes little (§5). Tuning the global scale τ\tau would increase the pressure toward sparsity and make the equation easier to read. Until a sparse, stable equation can be recovered, we interpret the dense one by ablation, removing one term at a time and measuring the change in forecast accuracy. In the learned frame, this reveals multiplicative gating terms, of the kind described for non-normal dynamics, and latents that consistently drive the dimensions most important for prediction.

Our claim on neural data is therefore not a sparse equation but an equation of known functional form. Unlike a neural drift, an explicit polynomial can be taken apart: whole classes of terms can be removed to test whether the dynamics need them. Because the readout bias fixes the latent origin, the latent frame is free only up to a linear map, which preserves polynomial degree, so these ablations do not depend on the frame. On MC-Maze (d=8d = 8), with each forecast started from the full model's state, removing the cubic terms makes the 10-bin forecast diverge (fp-bps −24.3), and removing the quadratic terms lowers it from +.122 to −.023, below simply holding the state fixed (+.076). Both nonlinear orders carry the forecast, and the cubic terms keep it bounded.

Recovering sparse equations from neural recordings remains a hard, open problem. The comparisons in §2 suggest how to make progress: two-stage pipelines give dense or unstable equations, while learning the equation inside inference, as SyLVER does, gives one that forecasts better than a neural state-space model of the same size. We therefore see a single-stage approach as the right foundation for this problem.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.