acceptodds
Under review as a conference paper at ICLR 2027

The Dynamics of Discoverability: How Trajectories and Priors Shape Equation Recovery

Abstract

How the dynamical regime of the observed system affects equation discovery has mainly been investigated through comparisons across systems. However, such comparisons vary both the equations and the dynamics, confounding the effect of the regime with the difficulty of recovering the equations symbolically. We separate the two by varying the forcing of Lorenz-84, moving its fully observed post-transient trajectories through fixed-point, periodic, and chaotic regimes while preserving the equations' functional form. Within each regime, we separately vary the amount of data, the noise, and prior knowledge of which terms the equations contain. We then measure how well two complementary approaches, sparse regression over a fixed library of candidate functions (SINDy) and an evolutionary search over symbolic expression trees (PySR), recover the true equations' terms and coefficients. We find that recovery depends on whether the sampled states distinguish combinations of candidate functions: equations remain poorly recovered from fixed-point data even when the candidate set contains only the true terms. We link the effect of the dynamics on both algorithms to one object: the moment matrix of the candidate functions under the invariant measure of the regime. Small eigenvalues mark weakly distinguishable combinations of candidate functions: we show that more data, less noise, and more prior knowledge can mitigate the resulting recovery difficulties, while a zero eigenvalue makes distinct equations indistinguishable on the visited states. Hence, a more precise prior needs less informative data, with consequences for data collection, method design, and evaluation.

Primary area:
learning on time series and dynamical systems
Keywords:
governing equation discovery, dynamical systems, system identification, scientific machine learning, symbolic regression, sparse regression, identifiability, invariant measures, chaotic dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JqLHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how dynamical regimes affect equation recovery with SINDy and PySR. The exprimental results on Lorenz-84 indicates that the critical factor of recovery is whether the sampled states distinguish candidate models. Chaos is neither necessary nor harmful for recovery under a fixed candidate library. Fixed-point data still yields poor recovery even with full priors. Further, they use eigenvalues of the moment matrix of candidate functions under regimes’ invariant measure to quantify how well candidate function combinations can be distinguished from observed trajectory data. Finally, the authors concluded that regime, data volume, noise and prior information jointly determine identification quality.

Critical strengths
  1. Controlled experimental design isolate the effect of dynamical regime on recovery performance. How regime, data volume, noise, and prior information affect identification quality is clearly revealed.

  2. The paper provides a theoretical explanation to identifiability for two very different algorithms. The minimum eigenvalue of the moment‑matrix can be computed before running discovery algorithms to assess identifiability.

Critical weaknesses
  1. The extent to which these results can be generalized to real‑world equation discovery is uncertain. Experiments are restricted to full‑state, noise‑controlled synthetic ODE data, but real-world measurements commonly face partial observation and process noise.

  2. The study only evaluates SINDy and PySR, two symbolic equation-discovery methods. A broader set of algorithms, including neural networks and nonparametric models such as Gaussian processes, remains unexplored. It is unclear whether the conclusions generalize to these alternative modelling paradigms.

Questions for authors
  1. Noise is treated as simple additive measurement noise in this work. How would structured noise or non-uniform sampling alter the recovery difficulty across different dynamical regimes?

  2. Appendix D demonstrates that transient data can enhance identifiability. Could the authors elaborate on this observation and provide deeper insights into why transient trajectories carry additional identifying information, and how this effect interacts with the dynamical regime under study?

1 comment
AuthorsOpenReview ↗

Response to Reviewer JqLH: realistic noise, more methods, irregular sampling, transients

W1, realistic noise. Our claim is that recovery is set by the moment-matrix spectrum. We tested whether this holds beyond white measurement noise. We ran all 8 L84 regimes with SINDy (Appendix I.6, Figure I.1) under three noise types: white and temporally correlated measurement noise at 1% (correlation time 10 samples), and process noise (stochastic dynamics). Each run is scored by γ_min, the paper's pairwise evidence γ = 𝓑·D_μ (Eq. 3.6) for the closest competitor that omits one true term; the floor is γ_min = 1. Under every noise type, no run below the floor recovers the equation (0 of 19), and most runs above it do (white 32/34, correlated 33/34, process 28/33). The noise level in 𝓑 is the derivative error measured against the true equation and corrected for autocorrelation, so γ_min is an oracle diagnostic. PySR gives the same result: 0 of 19 runs below the floor recover, and 99 of 101 above it do (white 34/34, correlated 34/34, process 31/33). Partial observation changes the question itself, since the equations must then be written in the observed variables; we leave it to future work.

W2, only SINDy and PySR. Figure 3 adds Operon, BMS, uDSR and ODEFormer (a pretrained transformer). A Gaussian process whose kernel is built from a term library is Bayesian linear regression on those terms (Rasmussen & Williams 2006, eqs. 2.8 to 2.12). Its posterior covariance over the coefficients is (N M/σ² + Σ_p⁻¹)⁻¹, so our moment-matrix analysis (Eqs. 3.4–3.5) applies unchanged (Appendix I.7). A GP with a generic kernel (e.g. RBF) returns no equation, so term recovery is not defined for it. Its posterior variance is small only near the visited states, so the same limit holds: the data constrain the model only where the trajectory goes.

Q1, structured noise and non-uniform sampling. In our framework, both change difficulty only through the pairwise evidence γ = 𝓑·D_μ (Eq. 3.6). The regime sets D_μ; noise type and sampling set 𝓑. Structured noise: see W1. Non-uniform sampling: in the benchmark (Figure 3; Appendix J), 252 of the 756 tasks drop 50% of the points at random. At matched noise this lowers recovery (SINDy 24% → 14%), but no method recovers any equation below the floor γ_min = 1, and recovery still rises with γ_min (five methods p ≤ 0.007; ODEFormer p = 0.07). On the L84 regimes (Appendix I.6, Figure I.1) the floor also holds (0 of 6 runs below it). Above it, recovery is lower than with uniform sampling (10/34 vs 32/34 successes): γ_min ≥ 1 is necessary but not sufficient here. PySR, given the same derivative estimates, recovers 34 of 34 runs above the floor under non-uniform sampling and 0 of 6 below it. So the data allow recovery above the floor; SINDy's lower rate there comes from the method, not the data.

Q2, why transients help. On the attractor, some combinations of library terms barely vary. These are the weak directions of the moment matrix M (small eigenvalues). The data cannot tell competitors that differ along them apart. Transients visit states off the attractor where those combinations do vary, so they raise exactly those eigenvalues (Appendix D.4, Proposition D.6). What matters is therefore where the new information lies, not how much there is. In a matched-budget test on all 8 L84 regimes (half the samples replaced by transients; Figure D.4), 97% of the information the transients add lies in the attractor's weak directions. The Soft F1 gain tracks that part (R² 0.71) and not the rest (R² 0.05). The regime enters only through how many weak directions its attractor leaves. A fixed point leaves almost all of them (gain +0.85 to +1.00), limit cycles leave some (+0.18 to +0.48), and chaos leaves few: +0.26 at N = 95 and none from N = 2114, where transients add nothing. PySR shows the same pattern: gains of +1.00 and +0.76 at the fixed point (N = 95, 2114), +0.23 and +0.05 on limit cycles, and +0.35 and 0 in chaos. At N = 95 the chaotic gain exceeds the limit-cycle gain; from N = 447 on (447, 2114, 10,000), the ordering is the same as SINDy's.

Reviewer_EYUEOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

An interesting study addressing: "Does chaos help or hurt equation discovery?" and other data regimes. They Use SYNDy (regression with fixed terms in eq) and PySR (evolutionary symbolic regression) to discover equations for two Lorenz systems in different parameter regimes: fixed point dynamics, limit cycles and Chaos. They find that generally learning is easier in the chaotic regime. They dig deeper and find the moment matrix of the candidate functions can help explain the effect. When its eigenvalues are small for the given data, the model can't distinguish some terms clearly and the equation recovery accuracy drops.

Critical strengths
  1. The soft F1 score seems like a good measure for equation discovery.
  2. The correlation between the dynamics regimes and, the moment matrix eigs and discovery performance is interesting and instructive.
  3. The consistent results across SINDy and PySR adds more credibility.
Critical weaknesses
  1. Only Lorenz systems. Even. double-pendulum, van der Pol or other simple systems that can have both periodic and chaotic regimes should have been considered. I understand the "removing confounders" argument in the discussions. But that doesn't excuse not doing the same study across multiple systems.
Questions for authors
  1. How much is the moment matrix result robust to the choice of basis used in SINDy and the PySR init library? Ex: Fourier vs polynomial.
  2. Would symmetries also cause spurious zero eigs in the moment matrix, e.g. rotation symmetry with sin and cos both in the sindy bank, would this yield zero eigs because a phase shift allows for degenerate ways of expressing the eq using sin and cos?
3 comments
AuthorsOpenReview ↗

Response to Reviewer EYUE: pendulum and other systems, Fourier basis, symmetries

W1, simplicity of the systems. The appendix also covers the jerk family (polynomial, rational and sin-rational; Appendix E, Figure E.1), and Figure 3 adds 63 ODEBench systems, including van der Pol and the pendulum (see also our reply to 2gqs, which adds Lorenz-96 up to 100D). As suggested, we also ran the full regime study on the driven damped pendulum, with the Lorenz-84 protocol unchanged (Section 5; 8 regimes: a fixed point, 2 limit cycles and 5 chaotic; Appendix F, Figure F.1). The regime ordering repeats: the fixed point never recovers, a limit cycle recovers with fewer samples than any chaotic regime, and the oracle library recovers everywhere except at the fixed point. Recovery again follows the paper's evidence γ_min (Eq. 3.6): Spearman ρ = 0.88 (data volume), 0.69 (noise) and 0.82 (prior quality). The pendulum is less noise-tolerant: with SINDy's paper settings, recovery fails beyond 2–5% noise in every regime. PySR on the same cells (Figure F.1E–H) also never recovers the fixed point (Soft F1 ≤ 0.15) and recovers every other regime with the oracle prior. Unlike SINDy, it recovers chaos with fewer samples than the limit cycles (0.99–1.00 in every chaotic regime from N = 2114, while the limit cycles stay at 0.77–0.81 up to N = 10,000), and it is even less noise-tolerant: in every regime its Soft F1 falls to 0.2–0.5 from 1% noise with finite differences and from 2% with Savitzky–Golay derivatives. At those noise levels the error of the estimated derivatives already exceeds their signal, so the data no longer separate the true terms from look-alikes such as sin ω for ω. The PySR indicator (Eq. 3.2) again follows recovery: Spearman ρ = 0.77 (data volume), 0.44 (noise) and 0.90 (prior quality).

Q1, Fourier basis. Here the basis is the library's function class. Adding sin/cos columns changes the functions the library spans, so the moment matrix (Eq. 3.4) and its eigenvalues change; a linear rewriting of the same library would keep its exact zeros. Our conclusions do not change. We refitted SINDy and PySR with polynomial and polynomial + sin/cos libraries on L84 and L96 (Appendix G.7, Figure G.1). For SINDy: (i) the regime ordering chaos > limit cycle > fixed point holds for every basis (e.g. L84 Soft F1 0.83/0.64/0.00 polynomial vs 0.70/0.55/0.00 Fourier-augmented); (ii) λ_min predicts recovery equally well in every basis (Spearman 0.83 to 0.86). The extra Fourier columns can lower recovery when they are nearly collinear with the polynomials (L84, −0.08 to −0.11), and the augmented spectrum explains 96% of that drop. For PySR, whose operators build composites such as x_j·sin(x_i), the same holds once the spectrum summary S is computed over the expressions it can reach and weighted by its complexity prior: S explains 85% (L84) and 96% (L96) of the drop in PySR's recovery (Figure G.1E, F).

Q2, symmetries. No: a symmetry of the equations makes eigenvalues of the moment matrix repeat, not vanish (L96 is cyclically symmetric, yet its chaotic moment matrices have λ_min from 10⁻³ to 10⁻², none zero). A zero eigenvalue needs two coefficient vectors that give the same function on the data; a symmetry alone does not create one, but it can confine the data so that such an identity holds on them. Such a zero arises in two ways (Appendix D.1, Remark 4): (i) Library redundancy. If the dictionary contains sin(θ₁−θ₂) next to sin θ₁ cos θ₂ and cos θ₁ sin θ₂, the identity sin(θ₁−θ₂) = sin θ₁ cos θ₂ − cos θ₁ sin θ₂ gives a zero eigenvalue for any data. The alternative expressions are the same function everywhere, so this zero is spurious for the dynamics. Removing the redundant columns eliminates it (2 exact zeros → 0 in our phase-oscillator test; Figure D.2B). (ii) Data that stay on the limit cycle. Hopf is rotation-symmetric, and its limit cycle is the circle x²+y² = 1. On that circle, Hopf and the harmonic oscillator are the same equation. Both fit the data perfectly (residual 10⁻¹²) but differ off the circle: Hopf attracts trajectories to the cycle, the oscillator does not. This gives 3 exact zero eigenvalues; SINDy returns a mix (Soft F1 0.30) and PySR the oscillator (10/10 runs). The zeros are real, not spurious: transients off the circle remove them, and SINDy then recovers Hopf (Soft F1 1.00; Figure D.2A); PySR does in 4 of 10 runs, and since the data now separate the two equations, its other failures come from its search.

Thank you for your response. The extra examples from other systems make the evidence stronger. The result is kind of narrow, but important, nevertheless.

Thank you for your reply and for engaging with our work. We would be grateful if you could tell us a bit more about what you find narrow in the result specifically, so we can address it further

Reviewer_2gqsOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper considers the prospects of model discovery: specifically under what conditions model discovery can be accomplished. This is done in the context of the SINDy and PySR algorithms. It is ultimately a deep dive into understanding under what conditions the recovery of governing equations can be made. Detailed simulations and experiments are made to validate the findings.

Critical strengths

This is a very nice and detailed study or when and how model discovery should be accomplished. It is thorough and well documented. That is the primary contribution of the work.

Critical weaknesses

It is a bit of a niche result. Although detailed and thorough, it lacks what I believe is a broad and compelling reason for ICLR. The big contributions and innovations are the symbolic algorithms themselves. Again, I appreciate the work, but think it would be a more appropriate contribution for a technical journal.

Questions for authors

The models are a bit simple, I think that much more complex dynamical systems should be considered. Is that possible?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 2gqs: significance and system complexity

W1, scope and contribution. We did not develop new algorithms. Equation discovery is not niche: physics and neuroscience build models with it, through symbolic regression (Brunton et al., 2016; Cranmer, 2023), pretrained transformers (d'Ascoli et al., 2024) and LLMs (Shojaee et al., 2025), and it is also the final step of other ML pipelines: symbolic distillation of neural networks (Cranmer et al., 2020), KANs (Liu et al., 2025), learned climate closures (Zanna & Bolton, 2020) and universal differential equations (Rackauckas et al., 2020). If the data cannot separate the true equation from the alternatives allowed by the prior, all of these pipelines fail. Indeed, our study works on the fundamentals of the topic: do the dynamical properties of the data carry enough information to determine the full physical representation of the target system? This is the identifiability question studied in causal and representation learning (Khemakhem et al., 2020; Locatello et al., 2019; Schölkopf et al., 2021), including for temporal data at ICLR (Yao et al., 2022), and classically for ODE models (Miao et al., 2011), asked here for learned dynamical models; our results thus give conditions for learnability, which any learning algorithm presupposes. The study has at least three practical consequences: (1) Benchmarks contain tasks the data cannot support: on ODEBench, where the true equations are known, our framework tells that 131 of 756 tasks, hence 25 out of 63 systems at some noise level, are impossible to recover. We tested it now for the rebuttal, across 6 different discovery algorithms, and none of them recovers the equations there (Section 3.5, Figure 3A). So, today's leaderboards, that should account for model failure, are polluted by data informativeness. (2) Predictive accuracy does not imply symbolic recovery, and hence mechanism retrieval. On those same tasks, 274 out of 786 runs reach R² > 0.99, but none recovers the equation. (3) The framework can give practical guidance on which data to collect, because recovery fails along specific combinations of library terms that barely vary on the attractor; for example, adding transients, which move the system off the attractor, makes these combinations vary and restores recovery (Appendix D.4, Figure D.4; same as the second question by JqLH). Algorithms are the engine; our contribution is to identify a fundamental limit they all share, arising from the data and how they use prior knowledge, in the same way that sample-complexity bounds complement learning algorithms.

Q1, simplicity of the systems. We added two tests. First, six algorithms on ODEBench add 63 systems (1D to 4D, four chaotic) to the paper's original five; the results are in (1) above (Figure 3). Second, SINDy and PySR on Lorenz-96 up to 100D (Appendix G.8, Figure G.2): Soft F1 falls with dimension (SINDy 0.93 to 0.23; PySR 0.99 to 0.78, mostly at small N). A fractional logit of Soft F1 on the spectrum summary S (the share of candidate terms the data determine, weighted by the method's prior; Appendix G.7), computed over the degree-two monomials of the variables each method receives, with forcing and noise as covariates, leaves dimension only 1.6% (SINDy) and 3.1% (PySR) of its effect. Fitted on D ≤ 10 alone, it predicts PySR's recovery within 0.02 at every D up to 100 (0.79 vs 0.78 observed at 100D); for SINDy it predicts 20D (0.59 vs 0.61) and is too pessimistic beyond (0.11 vs 0.23 at 100D). A library restricted to each variable's neighbours keeps Soft F1 at 0.94–0.95 (SINDy) and 0.98–0.99 (PySR) at every D. Our framework therefore extends to more complex systems in both senses: algebraically, with ODEBench's non-polynomial equations (trigonometric, exponential, rational), and in dimension. The same spectrum of M accounts for recovery in all of them.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General comment

We thank the reviewers for their careful reading. Their comments converge on one question: does the framework hold beyond SINDy, PySR and the Lorenz systems? To answer it, we added tests across methods, systems, bases, dimensions and noise models. All are in the revised PDF:

  • Section 3.5 and Figure 3 (new; all reviewers): six methods that share no mechanism (SINDy, PySR, Operon, BMS, uDSR, ODEFormer) on all 63 ODEBench systems, 756 tasks. On the 131 tasks where the data leave a true term below the noise floor, none of 786 runs recovers the equation, although half of them still predict the trajectory well (R² > 0.9). Above the floor, recovery rises with the evidence for every method.
  • Appendix D.1, Remark 4 (new; EYUE): symmetries and zero eigenvalues.
  • Appendix D.4 (extended; JqLH, 2gqs): why transients help, across all regimes.
  • Appendix F (new; EYUE): the full regime study on the driven damped pendulum.
  • Appendix G.7 (new; EYUE): libraries augmented with sin/cos, for SINDy and PySR.
  • Appendix G.8 (new; 2gqs): Lorenz-96 up to 100 dimensions.
  • Appendix I.5 (new): the evidence of the least resolved true term, γ_min, which defines the floor used in Section 3.5 and Appendices I.6 and J.
  • Appendices I.6 and I.7 (new; JqLH): correlated and process noise, irregular sampling, and Gaussian processes.
  • Appendix J (new): the ODEBench protocol and full results.
  • Main text (revised): the abstract and introduction now state the broader scope and link the work to symbolic distillation, Kolmogorov–Arnold networks, learned closures, universal differential equations, and identifiability. Sections 2 and 3 point to the pendulum, symmetry, transient and Gaussian process results. The Discussion and Limitations cover benchmark design and the two neural methods (uDSR, ODEFormer).
  • Figure 1 and Figure F.1: discrete colour keys, shared between the two figures.

Every test supports the paper’s claim: the dynamical regime, through the moment matrix, sets what the data can resolve, and that bounds what any method can discover. The updated supplementary zip contains the code to reproduce every new experiment and figure.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.