acceptodds
Under review as a conference paper at ICLR 2027

PDEDiscoveryBench: Evaluating Differential Equation Discovery from Spatiotemporal Data

Abstract

Discovering a partial differential equation (PDE) from observed fields requires choosing which terms may enter the law. Existing methods address this task, but published results rarely report how much prior information a method received, and to our knowledge no benchmark controls that information across method types. We introduce PDEDiscoveryBench, 337 ODE and PDE systems in one machine-readable format: 332 with reference equations and five unlabeled blood-flow fields simulated with computational fluid dynamics (CFD). Each labeled system is posed at three prior levels, from generic assumptions (L1) to equation-informed guidance (L3), written as natural-language prior cards that each method receives in its native form, and at four noise tiers. We evaluate seven sparse-regression, symbolic-search and large language model (LLM)-based methods by exact structural recovery and held-out normalized mean squared error (NMSE). Exact recovery is higher at L3 than at L1 for every method at every noise tier. Four case studies examine the prior from different sides: the data did not override a wrong prior card; LLM-PDESR recovered a KdV equation exactly only under its published, more specific specification; telling two LLM agents the noise model raised their exact recovery; and LLM agents given no physical description returned relations containing the Stokes balance for all five CFD fields. PDEDiscoveryBench makes prior information an explicit, controlled variable, so that new classical and LLM-based methods can be compared under the same priors and noise; we release its data, prior cards and scoring code. Our results suggest that discovery claims should state the prior behind them.

Primary area:
datasets and benchmarks
Keywords:
PDE discovery, equation discovery, benchmark, prior information, scientific machine learning, LLM agents, sparse regression, symbolic regression, AI for Science

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_nMZ7OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PDEDiscoveryBench, a benchmark for evaluating differential-equation discovery from spatiotemporal observations while explicitly controlling the amount of prior scientific information available to each method. The benchmark contains 337 systems, including 332 systems with reference equations and five unlabeled CFD blood-flow fields. The labeled collection spans ODEs and PDEs from multiple families, although a substantial fraction consists of 1D reaction–diffusion and conservation-law systems

Critical strengths
  1. Controlling scientific prior is a genuinely useful benchmark contribution. Existing benchmarks typically fix the prior implicitly, whereas PDEDiscoveryBench makes it an experimental variable and translates the same conceptual prior into each method’s native representation. This is a useful framework for comparing fundamentally different discovery paradigms.
  2. Large-scale evaluation across methods, priors, and noise. The benchmark contains 337 systems and evaluates seven methods spanning sparse regression, genetic/RL search, and LLM-based agents. More importantly, the paper does not merely report an aggregate leaderboard; it studies how results change as both prior information and noise change.
Critical weaknesses
  1. The claimed cross-method comparison is substantially confounded by unequal side information and budgets. This is my largest concern. SINDy and WSINDy receive the ground-truth number of terms K_{GT} as a support cap and undergo family/noise/prior-specific hyperparameter tuning, including clean inner-validation information for noisy tiers. Meanwhile, the LLM agents must infer temporal order and derivative estimators themselves, use retries, and have agent-specific budgets; DISCOVER and LLM-PDESR run once. The authors explicitly state that budgets are “method-native, not equalized.”
  2. Dataset diversity is considerably smaller than “337 systems” initially suggests.
Questions for authors

Could you please explain or address the weaknesses raised above?

4 comments
AuthorsOpenReview ↗

Response to Reviewer nMZ7

Thank you for recognizing the value of controlling prior information, and for pointing out the remaining differences between methods. We agree that these need to be clearer in the paper.

We checked whether our main findings remain after correcting these advantages. Prior information still improves recovery in most comparisons, with exceptions:

  • L3 beats L1 at every noise tier for SINDy, SGA-PDE, GPT-5.6-sol and Claude Opus 5.
  • WSINDy keeps the gain when it has the term-count cap and validates on clean or smoothed data; with raw noisy validation or without the cap, L3 ties or falls below L1 at 1% or 5% noise.
  • DISCOVER under noise is limited by the PINN its levels take derivatives from; they stay within 5.2 points (on clean data it gains, mostly through one family its L1 grammar cannot express).
  • On the first attempt, the agents gain 6.9–21.1 points.

W1. Unequal side information and budgets. We agree that these differences matter, but we think it is helpful to distinguish between method-specific settings and additional information provided by the benchmark.

For example, SINDy uses sparsity thresholds and regularization, WSINDy also needs to select weak-form integration settings, SGA-PDE uses population size and generations, while DISCOVER involves reinforcement-learning and PINN training. LLM agents, on the other hand, choose their own numerical procedures within a tool-call budget.

These settings do not have direct equivalents across methods. Even if we set every method to run for 100 iterations, one STLSQ iteration is very different from one evolutionary generation or one RL training iteration. We could impose a common time budget, but that would be a separate resource-controlled comparison rather than a common hyperparameter configuration.

Our goal is therefore to evaluate each method using its native procedure while making these computational differences explicit.

To make the tuning consistent, every sparse-regression and search method is now tuned on inner validation within the training data, and these tuned runs are our main results:

  • SINDy/WSINDy, as in the submission. With one shared configuration instead, L3 > L1 by 14.5–38.1 points (Appendix C.1, Table 13).
  • SGA-PDE, tuned on matching-noise validation: at most 1.3 points change in any cell.
  • DISCOVER, tuned per family and level (on clean data for the clean tier and on one 5% noise split for all noisy tiers): on clean data 41.0/45.2/54.2% vs. 41.9/47.3/58.7% with the published configuration. On noisy data, L1/L2 move by at most 1.8 points and L3 falls by 2.4–5.7 points, because configurations that fit noisy validation data better do not recover the structure better.

We also agree that some of our original settings provided additional information or attempts, beyond the prior cards:

  • Reference term count (K_GT). Removing it while keeping clean validation, SINDy's noisy L3 falls from 45.3/35.8/31.4% to 24.8/18.2/13.2% and WSINDy's from 84.0/63.5/51.6% to 40.3/27.7/26.7% (318 scalar PDEs). On clean data both fall below the agents (Table 11).
  • Clean validation on noisy tiers. Keeping the cap but validating on noisy data, SINDy changes little at 1% and 5% noise but falls to 21.1% at 10% (43.4/35.5/21.1% at L3, vs. 45.3/35.8/31.4%). WSINDy's strong-form validation score selects empty models on raw noisy data, so its L3 falls below L1 at 1% (13.8 vs. 14.8%). Smoothed validation gives 51.3/16.4/19.5%.
  • Retries. First-attempt results become the primary agent numbers. GPT-5.6-sol relied on retries: clean L3 falls from 74.1 to 51.8%. For Claude Opus 5, 380 noisy slots whose analysis tool failed before any answer were rerun, and the rerun counts as the first attempt; counting those slots as failures lowers 4 noisy cells by at most 8.1 points, and every L3 − L1 gain stays positive.
  • Temporal target. SINDy/WSINDy need it as an interface input; the agents infer it. The new table listing every information channel (Table 4) records this.

The submission removed the term count only on clean data (Table 11); the revision benchmarks the missing noisy setups (Table 12). WSINDy's lead under noise depends on the cap and clean validation, so we no longer claim that the original gaps establish relative noise robustness.

W2. Diversity of equation families. We agree that the family distribution matters. The 332 labeled systems form 19 families, and 175 are 1D reaction-diffusion or viscous conservation-law cases. We already report the family counts in Table 1 and family-weighted results in Table 10, but we agree that these findings were not sufficiently emphasized in the main discussion. The revised Results now give the family-weighted count next to the pooled one: L3 exceeds L1 in 21 of 25 method-tier cells with the 19 families weighted equally (22 of 25 pooled). The family-weighted L3−L1 interval excludes zero in only 8 of 25 cells (17 pooled), and the limitations now say so.

AuthorsOpenReview ↗

Looking forward to your feedback

Dear Reviewer nMZ7,

Thank you again for your constructive review. We hope our response and the revised PDF (changes in blue) have addressed your concerns. In particular, every sparse-regression and search method is now tuned on inner validation, the agents are scored on their first attempt, the noisy tiers are benchmarked without the reference term count and without clean validation (Table 12), and the Results now give family-weighted results next to the pooled ones.

As the discussion period ends on November 18 (AoE), please let us know if any concern remains, and we will answer it before then. If our response has resolved your concerns, we would be grateful if you would consider updating your score.

Best regards, The Authors

AuthorsOpenReview ↗

A note on the shared concerns

Dear Reviewer nMZ7,

We wanted to share one update before the discussion closes. Reviewer Ufr1 raised the same concern you described as your largest: SINDy and WSINDy receive the reference term count and clean inner validation, and the agents' retries are counted. After reading our response and the revision, Reviewer Ufr1 wrote that all their concerns had been well addressed and raised their score.

The changes that answered this concern apply equally to your review: every sparse-regression and search method is now tuned on inner validation within the training data, the agents are scored on their first attempt, and the noisy tiers are benchmarked without the reference term count and without clean validation (Table 12). On dataset diversity, the Results now give family-weighted results next to the pooled ones.

If any part of your concern remains, we would be glad to answer it before November 18 (AoE). Otherwise, we would be grateful if you would consider updating your score.

Best regards, The Authors

AuthorsOpenReview ↗

Final note before the discussion closes

Dear Reviewer nMZ7,

A brief final note, as the discussion closes today (November 18, AoE). Both other reviewers, whose reviews also questioned the side information and the fairness of the comparison, have now said that the revision addresses their concerns and raised their scores.

We understand this is a busy period. If you have any remaining question about the side-information ablation (Table 12), the tuning of every method on inner validation, or the family-weighted results, we would be glad to answer it today. If our response has resolved your concerns, we would be grateful if you would consider updating your score.

Best regards, The Authors

Reviewer_8FBdOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a new evaluation protocol for differential equation discovery methods, motivated by the fact that different methods are often given different forms of prior information. The proposed benchmark consists of 337 ODE and PDE systems and defines three prior levels. Seven discovery methods are evaluated.

Critical strengths

The benchmark covers several PDE discovery methods across different families. The motivation for making prior information a variable is meaningful.

Critical weaknesses
  1. The main motivation of the proposed benchmarks is to compare "classical and LLM-based methods across controlled levels of disclosed prior information for field input PDE discovery"; however, the proposed controlled prior might not fairly remove priors. The same prior level is translated into very different forms for different methods, and these translations may impose very different effective constraints. For example, for a closed-library method such as SINDy, specifying the candidate library directly removes many possible terms and strongly restricts the search space, while for an LLM, the same information is provided only as textual guidance and does not enforce the hypothesis space in the same way. In addition, beyond candidate-term selection, prior information may also enter the pipeline at other stages, for example, derivative estimation.

  2. The main conclusion of the analysis is somewhat trivial and expected: providing more prior information improves equation discovery. Therefore, I think the key value of the benchmark is not to demonstrate this trend but to provide a protocol to quantify and compare the amount of prior information used by different methods. However, I didn't see this part sufficiently demonstrated.

Questions for authors

Given the name "PDEDiscoveryBench", as well as the introduction and related work, the focus of the Benchmark is on PDE discovery, what is the reason for including several ODE systems in the Benchmark?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 8FBd

Thank you for these questions. They help us clarify what the prior levels are meant to control and what we can learn from them.

W1. Does the same prior level impose different constraints on different methods?

Yes. We provide the same scientific prior card to different methods, but we do not expect it to become exactly the same constraint in every algorithm. For example, if the card describes a source-free conservation law, SINDy can remove source terms from its candidate library, DISCOVER can restrict its candidate stems, while an LLM agent receives this information as a natural-language instruction. The methods may therefore use the same information differently, which is exactly what we want to study.

We therefore compare how methods respond to the same disclosed scientific guidance, rather than requiring them to have identical internal hypothesis spaces. We believe this is closer to how these methods would actually be used in practice.

The benchmark also measures how strongly each method is bound:

  • Compliance. The agents receive the card as unenforced text. Still, on the 318 scalar PDEs, 90.8–99.1% of their final L2/L3 equations stay within the card's terms, at every tier and for both agents. No out-of-span answer is exactly correct.
  • Derivative estimation. We agree it is a second entry point, and we treat it as part of the method: each method uses its own estimator, as a researcher would.

We also agree that we need to be more careful about information provided outside the prior cards. On the noisy tiers, SINDy/WSINDy selected their estimator with clean validation data and capped models at the reference term count. We now report results without both (reply to Ufr1). The revision adds a table listing every channel through which information reaches each method (Table 4).

W2. "More prior helps" is expected; quantify prior use instead.

We agree that better recovery with more correct prior information is generally expected. However, the consistent improvement across different methods and noise levels also supports the reasonableness and practical feasibility of our three-level prior design. More importantly, the extent of improvement varies substantially across methods, which is what we aim to investigate through our benchmark. The numbers below use our corrected main results (every tunable method tuned, agents on their first attempt).

  • Information per card. Counting the equation structures each card admits, L2 cards remove 4.2 bits and L3 cards 8.0 bits on average, relative to the generic L1 library (318 scalar PDEs; Appendix C.3).

  • Gain per bit differs across methods. Clean exact-recovery gain per bit at L3, in points:

    SINDyWSINDyClaude Opus 5GPT-5.6-solDISCOVERSGA-PDELLM-PDESR
    3.092.432.391.921.721.230.04

    SINDy gains 3.09 points per bit, LLM-PDESR almost none.

  • Uncertainty. With family-bootstrap intervals (19 families), the L3−L1 gain excludes zero in 17 of 25 method-tier cells pooled and 8 of 25 family-weighted (Table 10).

  • Prior vs. noise. On noisy tiers, prior level accounts for 9.1% of the variance in cell-level exact recovery and noise level for 7.2%, while method accounts for 73.7% (Appendix C.3).

  • More is not always better. Partial priors can hurt: WSINDy on clean data goes 74.4 → 72.3 → 94.9% (L1/L2/L3), and GPT-5.6-sol at 5% noise 3.0 → 1.5 → 13.0%. Under noise, DISCOVER's PINN derivatives limit every level, and L3 does not exceed L1. Wrong priors are not recovered from (Table 15).

We think these results make a more useful point than simply saying that L3 usually performs better: methods differ substantially in how they turn scientific guidance into recovered equations, and a more detailed prior can sometimes hurt. We will make this distinction clearer in the paper.

Q. Why ODEs?

We include 12 ODE systems among the 332 labeled cases because they are a limiting case of differential-equation discovery with no spatial derivatives. They also test whether a method can recognize the type and temporal structure of a problem without being given the governing equation. The agents, for example, are not told which type of case they are analyzing, which is why our title uses "differential equation discovery." To make the scope explicit, we also rename the benchmark DiffEqDiscoveryBench in the revision.

We also checked the results after removing the ODEs. On the remaining 320 PDEs:

  • no exact-recovery value moves by more than 2.8 points;
  • L3 > L1 in the same 22 of 25 method-tier cells as on the full set (the three exceptions are DISCOVER on noisy data);
  • the top two methods are the same in every cell, and the full ranking is unchanged in 9 of 12 cells (in the other three, methods less than one point apart swap or tie).

The revised PDF uses the new name and gives this table and the scope clarification in Appendix C.2 (Table 14).

AuthorsOpenReview ↗

Looking forward to your feedback

Dear Reviewer 8FBd,

Thank you again for your thoughtful review. We hope our response and the revised PDF (changes in blue) have addressed your concerns. In particular, the paper now lists every channel through which information reaches each method (Table 4), quantifies how much each method gains per bit of prior information (Appendix C.3), and shows that the results are essentially unchanged without the ODE systems (Table 14); the benchmark is renamed DiffEqDiscoveryBench to make its scope explicit.

As the discussion period ends on November 18 (AoE), please let us know if any concern remains, and we will answer it before then. If our response has resolved your concerns, we would be grateful if you would consider updating your score.

Best regards, The Authors

AuthorsOpenReview ↗

A note on the shared concerns

Dear Reviewer 8FBd,

We wanted to share one update before the discussion closes. Reviewer Ufr1 also questioned whether the same prior level puts every method on an equal footing. After reading our response and the revision, Reviewer Ufr1 wrote that all their concerns had been well addressed and raised their score.

For the two points in your review: Table 4 now lists every channel through which information reaches each method, including derivative estimation and tuning, so the effective constraint behind each prior level is explicit. Appendix C.3 quantifies how much each method gains per bit of prior information, so methods are compared by how well they use the same disclosed prior, not only by whether more prior helps. Table 14 shows the results are essentially unchanged without the ODE systems.

If any part of your concern remains, we would be glad to answer it before November 18 (AoE). Otherwise, we would be grateful if you would consider updating your score.

Best regards, The Authors

Thank you for the detailed rebuttal. The experiments addressed my main concern, I have raised my score to positive. Good luck!

Reviewer_Ufr1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper introduces PDEDiscoveryBench, 337 ODE/PDE systems: 332 with reference equations and five unlabeled stationary CFD blood-flow fields. Each labeled system has prior cards at three levels, which are translated into the native input of seven methods (SINDy, WSINDy, SGA-PDE, DISCOVER, LLM-PDESR and two LLM agents) and evaluated at four noise tiers; LLM-PDESR is run on clean data only. Methods are scored by exact structural recovery and held-out NMSE. The main findings are that exact recovery is higher at L3 than at L1 for every method and tier, that noise sharply reduces recovery, and that low NMSE does not imply the correct structure.

Critical strengths
  1. Making the prior explicit through prior cards and crossing three prior levels with four noise tiers is a useful protocol: it turns an implicit and usually unreported choice into a controlled variable of the benchmark.
  2. Reporting exact structural recovery alongside NMSE separates equation correctness from numerical fit. The term-count ablation (Table 10) shows why low numerical error alone is insufficient, and the card-swap and noise-disclosure case studies (Tables 11 and 13) illustrate sensitivity to wrong priors.
Critical weaknesses
  1. The L1-versus-L3 contrast, the paper's headline result, changes more than the prior. SINDy and WSINDy are re-tuned per prior level (Appendix B.3, B.5), and on noisy tiers DISCOVER switches from a PINN run with the generic token list at L1 to Mode 2 with a different network, sampling fraction and number of rounds at L2/L3 (Appendix B). The gains attributed to prior disclosure therefore mix in configuration changes, which weakens the benchmark's main claim.
  2. Cross-method comparisons are not on equal footing. SINDy and WSINDy receive the reference term count K_GT as a support cap and use clean inner-validation data, the agents report cumulative success over retries, and budgets are method-native (Sec. 4.2). The paper discloses these asymmetries, but disclosure does not remove them: the method ranking under noise in Fig. 2 cannot be read as relative robustness, and the K_GT cap is ablated on clean data only (Table 10).
Questions for authors
  1. (Clarification) Can L1 and L3 be compared for DISCOVER under one identical configuration on the noisy tiers, so that the gap reflects the prior alone?
  2. (Clarification) Does the cross-method ranking under noise persist when SINDy and WSINDy run without the K_GT cap and with matching-noise validation, and when agents are scored on their first attempt?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Ufr1

Thank you for looking closely at our experimental settings. We agree that prior information and numerical configuration should be separated. Following your comments, we ran additional experiments to examine these effects; we address the findings below.

W1 / Q1. Configuration changes inside the L1-vs-L3 contrast.

SINDy/WSINDy. We apply the same tuning procedure at all three levels: the same search grid, validation rule, and selection criterion. The selected hyperparameters can still differ because the candidate library changes with the prior. For example, on a heat equation, SINDy's L1 library has 67 candidate terms, while L3 has only two. A sparsity threshold selected for the larger library may not be the best choice for the two-term library. In practice, we would normally retune when the candidate library changes.

To check whether retuning explains the improvement, we ran two controls:

  • L3 with L1's settings. Refitting L3 with the L1-selected hyperparameters still beats tuned L1 in all 8 method-tier cells (e.g. SINDy 86.8 vs 67.9% clean, WSINDy 62.9 vs 50.0% at 1%; Appendix C.1, Table 13).
  • One shared configuration. With one configuration for every family, tier and level, L3 beats L1 by 14.5–19.2 points (SINDy) and 21.1–38.1 (WSINDy) in every cell.

DISCOVER. We agree with your concern here. In the submitted noisy experiments, L1 used different numerical settings from L2/L3, making it difficult to separate the effect of prior information from the configuration itself. We therefore reran L1 using the same numerical settings as L2/L3, so that the candidate grammar is the main difference between levels. Because we tune SINDy, WSINDy and SGA-PDE, we also tuned DISCOVER (per family and level, on noisy validation data: one 5% noise split, reused at every tier). These tuned runs are now our main DISCOVER results:

NoiseL1 submittedL1 matchedL1 / L2 / L3 matched and tuned
1%31.332.834.0 / 38.0 / 32.8
5%21.726.827.1 / 26.2 / 22.9
10%16.016.614.8 / 16.0 / 14.2

All values are exact-recovery percentages over 332 labeled systems. In the submission, DISCOVER's noisy L3 − L1 gain was +4.8/+6.9/+0.6 points. With matched settings it becomes +3.3/+1.8/0.0, and after tuning −1.2/−4.2/−0.6. Under noise, every level takes its derivatives from a PINN fitted to the noisy field, so we attribute this to the PINN limiting recovery at every level: the three levels stay within 5.2 points of each other at every tier. Tuning also selects configurations that fit noisy validation data better, which does not improve structure. The revision explains this next to DISCOVER's noisy results. On clean data, the tuned L3 still exceeds L1 pooled (54.2% vs. 41.0%), but not with the 19 families weighted equally (29.2% vs. 29.5%, Table 10). Most of that clean gain comes from one family: 25 of DISCOVER's net gain of 44 systems are trigonometric-flux conservation laws (0 at L1, 25 at L3), which its L1 token list cannot express. Without that family, the net gain is 19 systems. The DISCOVER appendix now says so.

W2 / Q2. Does the noisy ranking persist?

The known term count and clean validation data gave SINDy/WSINDy additional information, while retries gave the agents additional attempts. The submission removed the term count only on clean data (Table 11), so we benchmarked the missing noisy setups, removing each separately (318 scalar PDEs, L3 at 1/5/10%, each arm retuned):

  • SINDy: after removing the term-count cap and using noisy validation, recovery changes from 45.3/35.8/31.4% to 21.4/19.5/3.5%. Using noisy validation alone gives 43.4/35.5/21.1%. L3 stays above L1 in every setting.
  • WSINDy: on raw noisy validation data, its strong-form residual criterion tends to select an empty model, yielding 13.8/15.7/17.0%, and L3 falls below L1 at 1% noise (13.8% vs. 14.8%). Smoothing validation data gives 51.3/16.4/19.5% with the term-count cap, and 21.7/1.3/7.9% without it. Without the cap, L3 ties L1 at 5% and falls below it at 1% with raw validation (0.0% vs. 3.5%). The validation procedure matters a great deal for WSINDy.
  • LLM agents: using only the first attempt, L3 − L1 stays positive at every tier (clean/1/5/10%):
    • GPT-5.6-sol: +15.4/+14.2/+9.9/+6.9;
    • Claude Opus 5: +19.0/+21.1/+15.1/+16.6.

The benchmark is meant to measure how each method uses the prior, not to rank methods, so we report these as additional setups. In them, WSINDy's lead under noise does not persist (DISCOVER has the highest recovery in 8 of 9 noisy cells, Claude Opus 5 in 1), so we qualified the robustness reading of Figure 2: WSINDy leads only with the cap and clean validation, and the Results and conclusion now say so. A new "Side information" paragraph in the Results summarizes these setups, and a new appendix table (Table 12) gives every arm.

AuthorsOpenReview ↗

Looking forward to your feedback

Dear Reviewer Ufr1,

Thank you again for your careful review. We hope our response and the revised PDF (changes in blue) have addressed your concerns. In particular, DISCOVER's noisy L1 now uses the same numerical configuration as L2/L3, two controls (L3 refit with L1's hyperparameters, and one shared configuration) show that SINDy's and WSINDy's gains do not come from retuning (Table 13), and the noisy tiers are now benchmarked without the term-count cap, with noisy validation and with the agents' first attempts (Table 12).

As the discussion period ends on November 18 (AoE), please let us know if any concern remains, and we will answer it before then. If our response has resolved your concerns, we would be grateful if you would consider updating your score.

Best regards, The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers

Thank you for the detailed comments. We agree that the fairness of the comparison needs to be explained more clearly, and the reviews helped us identify several settings that needed to be corrected.

Our goal is to control what scientific prior is disclosed to each method, while allowing each method to use that prior in its own way. At each prior level, every method receives the same scientific content: the same candidate terms and mechanisms. Each method uses it in its native form (a library for SINDy, a grammar for DISCOVER, text for the agents). Each is configured as a researcher would configure it, with its own hyperparameter tuning and derivative estimation. Methods differ in how much they gain from the same information and in which problems they handle well. This difference is actually one of the things we want to study. We aim to provide the same scientific guidance while allowing each method to use it in its natural way, rather than forcing all methods to follow the same procedure.

We also agree that some of our original experimental settings gave certain methods additional information or advantages, or were not applied consistently:

  • SINDy/WSINDy also received the reference term count and clean validation data on the noisy tiers.
  • The agents' retries were counted.
  • DISCOVER's noisy L1 runs used a different configuration from its L2/L3 runs.
  • SINDy and WSINDy were tuned, but SGA-PDE and DISCOVER were not.

Following these concerns, we conducted additional experiments and re-evaluated the affected results. Every sparse-regression and search method is now tuned on inner validation within the training data (LLM-PDESR keeps its upstream configuration), and the agents are scored on their first attempt. We found that:

  • Prior information still matters. L3 improves exact recovery at every noise level for SINDy, SGA-PDE (by 1.0–2.3 points under noise) and both agents, and for WSINDy when it keeps the term-count cap and validates on clean or smoothed data.
  • Under noise, DISCOVER is limited by its derivative estimates. Once its L1 configuration matches L2/L3, every level takes derivatives from a PINN fitted to the noisy field, and its three levels stay within 5.2 points of each other.
  • We benchmarked the missing setups. The submission removed the reference term count only on clean data (Table 11); the revision adds the noisy tiers without it and without clean validation (Table 12).

We explain the changes in the individual replies below. The revised PDF updates Figure 2 and Tables 9 and 10 to these results, reports DISCOVER's noisy result in the abstract, Results and conclusion, and adds a table for the side-information ablation (Table 12).

Update (Nov 16). We have uploaded a new version of the revised PDF in which all text changed since the submission is shown in blue (deleted text is omitted). The content is the same as the revision posted on Nov 7. Figures 2 and 5 were also redrawn with the updated results, and Figures 3 and 4 relabel the blood-flow systems as 0333–0337; these figure changes are not colored.

Dear Reviewer Ufr1,

Thank you for reading our response and the revised PDF, and for raising your score. We are glad that the additional experiments resolved your concerns about the L1-versus-L3 contrast and the cross-method comparison; your review led directly to the tuning controls (Table 13) and the side-information ablation on the noisy tiers (Table 12).

If anything still keeps you from a stronger recommendation, for example in the presentation or in what the benchmark lets readers conclude, we would be glad to address it before the discussion period closes on November 18 (AoE).

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.