- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates how dynamical regimes affect equation recovery with SINDy and PySR. The exprimental results on Lorenz-84 indicates that the critical factor of recovery is whether the sampled states distinguish candidate models. Chaos is neither necessary nor harmful for recovery under a fixed candidate library. Fixed-point data still yields poor recovery even with full priors. Further, they use eigenvalues of the moment matrix of candidate functions under regimes’ invariant measure to quantify how well candidate function combinations can be distinguished from observed trajectory data. Finally, the authors concluded that regime, data volume, noise and prior information jointly determine identification quality.
Critical strengths
-
Controlled experimental design isolate the effect of dynamical regime on recovery performance. How regime, data volume, noise, and prior information affect identification quality is clearly revealed.
-
The paper provides a theoretical explanation to identifiability for two very different algorithms. The minimum eigenvalue of the moment‑matrix can be computed before running discovery algorithms to assess identifiability.
Critical weaknesses
-
The extent to which these results can be generalized to real‑world equation discovery is uncertain. Experiments are restricted to full‑state, noise‑controlled synthetic ODE data, but real-world measurements commonly face partial observation and process noise.
-
The study only evaluates SINDy and PySR, two symbolic equation-discovery methods. A broader set of algorithms, including neural networks and nonparametric models such as Gaussian processes, remains unexplored. It is unclear whether the conclusions generalize to these alternative modelling paradigms.
Questions for authors
-
Noise is treated as simple additive measurement noise in this work. How would structured noise or non-uniform sampling alter the recovery difficulty across different dynamical regimes?
-
Appendix D demonstrates that transient data can enhance identifiability. Could the authors elaborate on this observation and provide deeper insights into why transient trajectories carry additional identifying information, and how this effect interacts with the dynamical regime under study?
Response to Reviewer JqLH: realistic noise, more methods, irregular sampling, transients
W1, realistic noise. Our claim is that recovery is set by the moment-matrix spectrum. We tested whether this holds beyond white measurement noise. We ran all 8 L84 regimes with SINDy (Appendix I.6, Figure I.1) under three noise types: white and temporally correlated measurement noise at 1% (correlation time 10 samples), and process noise (stochastic dynamics). Each run is scored by γ_min, the paper's pairwise evidence γ = 𝓑·D_μ (Eq. 3.6) for the closest competitor that omits one true term; the floor is γ_min = 1. Under every noise type, no run below the floor recovers the equation (0 of 19), and most runs above it do (white 32/34, correlated 33/34, process 28/33). The noise level in 𝓑 is the derivative error measured against the true equation and corrected for autocorrelation, so γ_min is an oracle diagnostic. PySR gives the same result: 0 of 19 runs below the floor recover, and 99 of 101 above it do (white 34/34, correlated 34/34, process 31/33). Partial observation changes the question itself, since the equations must then be written in the observed variables; we leave it to future work.
W2, only SINDy and PySR. Figure 3 adds Operon, BMS, uDSR and ODEFormer (a pretrained transformer). A Gaussian process whose kernel is built from a term library is Bayesian linear regression on those terms (Rasmussen & Williams 2006, eqs. 2.8 to 2.12). Its posterior covariance over the coefficients is (N M/σ² + Σ_p⁻¹)⁻¹, so our moment-matrix analysis (Eqs. 3.4–3.5) applies unchanged (Appendix I.7). A GP with a generic kernel (e.g. RBF) returns no equation, so term recovery is not defined for it. Its posterior variance is small only near the visited states, so the same limit holds: the data constrain the model only where the trajectory goes.
Q1, structured noise and non-uniform sampling. In our framework, both change difficulty only through the pairwise evidence γ = 𝓑·D_μ (Eq. 3.6). The regime sets D_μ; noise type and sampling set 𝓑. Structured noise: see W1. Non-uniform sampling: in the benchmark (Figure 3; Appendix J), 252 of the 756 tasks drop 50% of the points at random. At matched noise this lowers recovery (SINDy 24% → 14%), but no method recovers any equation below the floor γ_min = 1, and recovery still rises with γ_min (five methods p ≤ 0.007; ODEFormer p = 0.07). On the L84 regimes (Appendix I.6, Figure I.1) the floor also holds (0 of 6 runs below it). Above it, recovery is lower than with uniform sampling (10/34 vs 32/34 successes): γ_min ≥ 1 is necessary but not sufficient here. PySR, given the same derivative estimates, recovers 34 of 34 runs above the floor under non-uniform sampling and 0 of 6 below it. So the data allow recovery above the floor; SINDy's lower rate there comes from the method, not the data.
Q2, why transients help. On the attractor, some combinations of library terms barely vary. These are the weak directions of the moment matrix M (small eigenvalues). The data cannot tell competitors that differ along them apart. Transients visit states off the attractor where those combinations do vary, so they raise exactly those eigenvalues (Appendix D.4, Proposition D.6). What matters is therefore where the new information lies, not how much there is. In a matched-budget test on all 8 L84 regimes (half the samples replaced by transients; Figure D.4), 97% of the information the transients add lies in the attractor's weak directions. The Soft F1 gain tracks that part (R² 0.71) and not the rest (R² 0.05). The regime enters only through how many weak directions its attractor leaves. A fixed point leaves almost all of them (gain +0.85 to +1.00), limit cycles leave some (+0.18 to +0.48), and chaos leaves few: +0.26 at N = 95 and none from N = 2114, where transients add nothing. PySR shows the same pattern: gains of +1.00 and +0.76 at the fixed point (N = 95, 2114), +0.23 and +0.05 on limit cycles, and +0.35 and 0 in chaos. At N = 95 the chaotic gain exceeds the limit-cycle gain; from N = 447 on (447, 2114, 10,000), the ordering is the same as SINDy's.