acceptodds
Withdrawn from ICLR 2027

PriSIFT: Prior-Fitted Structural Inference from Trajectories for Sparse ODE Discovery

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Sparse ODE discovery requires identifying a compact governing structure from an exponentially large space of candidate supports. Existing methods solve this structural selection problem independently for each new system, while exhaustive evaluation becomes increasingly costly as the candidate library grows. We introduce PriSIFT, a prior-fitted framework that learns transferable structural regularities from synthetic ODE systems and reuses them to guide discovery on unseen dynamics. Given observed trajectories, a Set-Time Transformer produces term-level structural scores in a single forward pass; these scores define a compact set of plausible support proposals, which are then fitted and verified using task-local extended-BIC rather than a learned final selector. This separation allows cross-system structural knowledge to be amortized while keeping the final scientific decision grounded in the observed data. On a 20-system external literature benchmark held out from method development, PriSIFT achieves the strongest dimension-averaged macro support F1 among evaluated methods (0.678). In matched compute experiments on the fresh holdout, it evaluates 154× fewer candidate supports and achieves a 69× measured wall-clock speedup over exhaustive EBIC. These results suggest that prior-data fitting can provide transferable structural guidance for efficient and reliable equation discovery.

Primary area:
learning on time series and dynamical systems
Keywords:
Prior-data fitted networks, ODE discovery, sparse system identification, amortized search

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Ff6eOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents a framework for ODE discovery by training transformers on ODE trajectories. The framework works by training a transformer on synthetic trajectories of ODEs and then using that to constrain the search space of the terms and then fitting those to the observed data. They evaluate on 21 held out systems after training on 20 and tuning on 80 systems from public benchmarks.

Critical strengths

None

Critical weaknesses
  1. Its tough to establish any novelty as the paper combines existing methods without stating what is the contribution beyond this integration of prior methods.
  2. Its unfair to compare their method with EBIC and claim compute reduction as PriSIFT requires training while EBIC does not especially since there is modest boost in accuracy.
  3. The paper is extremely hard to follow. It is riddled with presentation errors such as inconsistent names of methods and axis labels in figures.
  4. The method is referred by various names across the entire paper: ODE-PFN in Figure 1 and BayesODE-PFN-Search in Figure 4.
  5. Lack of evidence for the claimed structural regularization mechanism contribution.
  6. RQ3: PriSIFT's support F1 improvement over EBIC (0.678 vs. 0.666) is statistically indistinguishable.
  7. Figure 3a uses Development and Untouched P9, while the main text refers to fresh holdout and 20-system literature corpus. The connection between these labels is not explained.
Questions for authors

See weakness

Reviewer_jipCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The submission proposes a method for learning ODEs under the assumption that the vector field is a sparse linear combination of a fixed library. It proposes to amortize the prediction of the support, i.e., which of the basis function are present in the vector field, via a pre-trained transformer-based architecture delivering logits. These are treated as proposals for which terms to include. The method then uses an extended BIC selection procedure an ridge regression to fit the actual coefficient values per trajectory.

Critical strengths

Some parts of the exposition regarding the high level idea are fairly straight forward to read and comprehend.

Critical weaknesses

I have multiple concerns about this submission.

  1. Novelty limitations: Many existing works (including ODEFormer) pose the problem in two stages consisting of structural candidates (e.g., ODEFormer also samples candidate structures as proposal structures) and selection + coefficient learning (again, ODEFormer and previous works it cites, also optimizes coefficients via an L2 loss across different sampled candidates and selects the optimal one). Of course, the scope for ODEFormer is quite different as it is not limited to linear combinations of basis functions (see next point). The idea of using amortized pre-training to select which candidates among a final library are present in a given expression with a second stage to actually select and form the expression is extremely common. For example, [1] from 2016 does just that to compose programs. I understand that this is not directly related, but my main claim is that conceptually as well as methodologically, there is little novel insight. There is also no principled/theoretical motivation for this approach, i.e., it is a plausible idea to try, but whether it is a good idea entirely hinges on empirical performance (see next point).

I would also disagree with some specific statements: l.131 "providing no reusable, composable notion of candidate quality: a generator that misses one term..." Other methods to provide multiple proposals that are sampled from their internal distribution of plausible candidates. Since they are often not restricted to linear combinations of a fixed library (e.g., ODEFormer can nest non-linearities and combine them non-additively, etc) they don't have the strict notion of "an omitted term". However, even in linear combinations, it is not clear that "missing a term" is the relevant failure mode. In many ways, sampling entire compositions of terms according to model likelihoods is possibly more relevant than mere term selection.

[1] Balog, M., Gaunt, A. L., Brockschmidt, M., Nowozin, S., & Tarlow, D. (2016). Deepcoder: Learning to write programs. arXiv preprint arXiv:1611.01989.

  1. Scope limitations: The chosen scope on which the approach is tested and evaluated is too narrow. After some searching in the appendix, my current understanding is that the pre-trained support predictor is trained only on 2D systems with up to 4-th degree polynomials as basis functions. Within the scope of the existing literature, this is way too narrow. It also does not cover most of the chosen evaluation datasets such as ODEBench and MDBench (non-polynomial, many 3D equations, etc), which makes me wonder how this evaluation was even performed. There is a variety of other concerns in terms of too narrow training/evaluation choices such as the fixed regularization parameters in the loss (eq. 6), a single additive noise model with a single fixed variance (multiplicative and various variances are standard), not varying the number of 32 observations (most of this without justification, selection protocols, or ablations).

  2. Evaluation: This is one of the main points. In "4.1 Corpora", the submission claims three public benchmarks.

  • Strogatz: cites the Strogatz book, not the Strogatz dataset; which one/version is used here
  • ODEBench: The cited work (Gilpin, 2021) does not introduce a dataset and does not contain the word ODEBench. I assume the authors meant to cite ODEFormer (d'Ascoli, 2023) which actually introduced the ODEBench dataset (this work was in ICLR 2024 however; in general it would be good to update the citations to where they actually appeared instead of the arxiv references)? ODEBench contains most of the systems from the Strogatz dataset as well (different initial conditions, but same systems).
  • MDBench: The MDBench paper presents a benchmark consisting of PDEs and ODEs. Since PriSIFT only works for ODEs I assume that only the ODE part of MDBench is used? However, MDBench literally just copies the entire ODEBench as their ODE part.

Moreover, a substantial fraction of the systems in ODEBench are 3 or 4 dimensional, which is not covered by the pre-trained prior network. Is the evaluation restricted to 2D systems? How big is the overlap between Strogratz and ODEBench? With all these missing details (are those also sub-sampled to 32 observations, what are chosen noise levels, etc, etc), I cannot judge the reported empirical performance.

Finally, I could not find any concrete information about the claimed "20-system literature corpus curated ..." What are these systems, how have they been curated, from what sources exactly (only Lüders, 2022 is mentioned without further details). Appendix A.2 also only says 3 sentences about this including merely stating "seven canonical literature systems" and, oddly, that the "evaluation seeds 421000-421009 were frozen independently of the development seeds."

  1. Clarity: Large parts of the paper are extremely hard to follow. There are many verbal constructions that I cannot map to precise mathematical/modeling concepts. Unnecessary LLM-style contrasts and odd implementation references while at the same time lacking mathematical descriptions of relevant contents make this hard to judge and read. Just examples: in eq. (1) what are these losses (some are in the Appendix, but it would still be good to understand it in the main). In section 3.3 how does the point embedding work? It is also unclear to me in this part whether trajectories refer to different components of a single multi-variate trajectory, or trajectories from the same system for different initial conditions, or trajectories from different ODEs. In the latter case, the trajectory pooling does not make sense to me? In the Baselines. paragraph in 4.1, what does it mean that baslines are "converted into proposal heuristics". Are they not used as intended for direct system inference, but somehow squeezed into the existing proposal/fitting split? How? Why? Artificial unnecessary contrasts like the one in l.319 confuse more than they help. Finally, the Appendix exposes really awkard LLMy unnecessary (and confusing) code/implementation references. I don't think the checkpoint naming convention (l.639), the used seeds, or the specific SHA256 hashes of the "untouched corpus manifest" or the "degree-fice specialist" are really relevant in the paper.
Questions for authors

See Weaknesses.

Reviewer_wzpKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a new method for ODE discovery. In particular, it tries to find a sparse representation of the system (as in SINDy), but it uses a prior-fitted network to select the most promising sets of terms. Each of those candidates is then fit to the data to estimate the term coefficients. The final selection is performed by computing an extended Bayesian information criterion. The network is trained on a synthetic dataset of ODEs. The training tasks involve predicting the support (i.e., which terms should be included), the values of the coefficients, and the actual vector field for a better supervision signal. The paper shows that the proposed method (PriSIFT) can match the performance of exhaustive search while being much less computationally expensive. It can also achieve much better performance than SINDy-related methods.

Critical strengths
  • The model offers a computationally efficient way to find a sparse ODE representation of the system without an exhaustive search.
  • It performs better than standard SINDy approaches.
  • The paper is well written, and the method is clearly described.
Critical weaknesses

My main concern is that the problem seems a bit toyish. We assume that we already know that the ODE is represented as a linear combination of terms from a predefined (small) library. We just don't know exactly which terms these are. I understand that SINDy (even with its selection algorithms) may not perform well if the library is much bigger than necessary. Still, it seems to me that some simple improvements such as heuristics or regularizations would significantly close this gap. We can already see from the experiments that MIOSR optimization and Bayesian-SINDy give very good performance in terms of NRMSE. I'm wondering whether running SR3 or STLSQ with different parameters or smaller libraries (maybe only up to 3-degree or 2-degree terms) would significantly improve performance. Fourth-degree terms do not appear that often in real-world settings either way. Or perhaps running SINDy for a few random samples of the support? I understand that there are a lot of possible subsets for the support (2^15). But SINDy already does feature selection. For instance, maybe we can just check all five-term subsets? I'm just not sure if this current instantiation of the problem (with only 15 terms in the library) actually warrants large-scale pretraining. I think a much more realistic problem is not knowing which terms to include in the library in the first place.

  • The approach currently works only for 2D ODEs.
  • The library is fixed and small - only 15 polynomial terms.
  • No comparison with ODEFormer - this is one of the most relevant baselines because it also uses large-scale pretraining and a transformer architecture to predict the equation structure.
  • I am not sure the experiments with degree 5 terms are fair. PriSIFT cannot predict fifth-degree terms, so it's not a surprise that it performs better. If you decide to exclude fifth-degree terms from the problem formulation, then they should not be added back to other methods. The same decision should hold for all methods.
  • There are some missing details about the training. How many ODEs were used during pre-training? How much time did it take?
Questions for authors
  • In Section 4.1, you say that you use ODEBench, but you cite a different paper. ODEBench was proposed by d'Ascoli et al. (2024)
  • Traditional SINDy already performs iterative term selection. I'm wondering if some simple search heuristic, like running SINDy for some number of randomly selected subsets of features, might give comparable results.
  • Have you tried running SINDy with different hyperparameters to check if stronger regularisation or different thresholds improve the performance?
  • Can you include a clear table/figure showing how the performance and computation time look for all methods and potentially how those things scale as the search space is varied?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.