- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper evaluates whether four general-purpose multimodal LLMs can propose differential-equation structures and numerical coefficients without feedback or external coefficient correction. It studies ten systems across twelve configurations, with data-withheld, prompt, noise, and regression comparisons. Five candidates per run are Pareto-screened, and success means a retained candidate matches the true terms and coefficients within 10%.
Critical strengths
-
A substantial empirical sweep separates structure recovery from coefficient accuracy across four models, ten systems, and twelve configurations, with data-withheld, prompt, and noise controls.
-
Detailed appendices examine failure stages, threshold and screening sensitivity, statistical uncertainty, regression retuning, and non-identifiability. The paper openly reports important limitations rather than hiding them.
Critical weaknesses
-
The motivation and incremental contribution need sharper justification. Isolating a model’s unaided capability is a legitimate research question, but this setting alone does not establish substantial novelty or practical value. LLM-SR already studies full-equation generation and removal of iterative refinement in separate ablations; the paper should clarify what new understanding its combined protocol adds. Likewise, retaining correct candidates with two existing screening criteria is a useful auxiliary result, but does not establish reliable final selection. The contribution therefore rests primarily on the empirical insights.
-
Those insights remain insufficiently deep and generalizable. Data helping, missing terms blocking recovery, and candidate generation differing from selection are largely intuitive; detailed counts are useful, but the study does not sufficiently test their causes or derive validated guidance. Written-reasoning categories do not establish the actual computation, and no-data failures do not exclude data-cued recognition. Randomized non-round coefficients and independent solutions would strengthen the inference claim; missing-term hints, aligned derivative inputs, and same-model fitting/feedback comparisons would test the proposed bottlenecks. Without such evidence, the results mainly characterize behavior on the chosen instances rather than explain discovery capability more broadly.
Questions for authors
-
What important uncertainty does the unaided setting resolve, and what precisely extends LLM-SR’s related ablations? Which new understanding, rather than the protocol combination itself, is the main contribution?
-
Can randomized non-round coefficients, transformed equations, and independent solutions distinguish data-based inference from recognition of familiar laws?
-
Which findings yield validated practical guidance? Could missing-term hints, pointwise-aligned derivatives, and matched fitting/feedback arms test the explanations for failure and input effects? Please also distinguish candidate retention from final selection.
What models contribute before fitting and how to use their proposals
Thank you for the careful reading. We revised the Introduction to explain why measuring a model’s own coefficient estimates matters even when a researcher ultimately plans to fit the equation numerically.
About relation to LLM-SR:
LLM-SR tested the effects of removing iterative refinement and external coefficient fitting in separate experiments, measuring prediction error. We removed both together and checked the terms and coefficients the model wrote. On fresh observations, a model could propose the correct terms and adjust its coefficient estimates before an optimiser or feedback could influence its answer (Section 4.3). LLM-SR, Section 3.5
About what the fresh trajectories showed:
We kept the equation families familiar and generated new observations with changed coefficients. For example, DeepSeek identified the correct terms on both the wave and Kuramoto–Sivashinsky systems. Its estimates followed the new values for the wave equation but stayed near the published values for Kuramoto–Sivashinsky. Recognising the equation form could therefore coexist with estimating a coefficient. The written explanations leave the model’s internal calculation unresolved. The experiment used fixed coefficient shifts and new initial conditions. Continuous sampling, transformed equations and unfamiliar structures remain open (Figure 3).
About using a proposed equations:
A researcher can ask for equations complete with numerical coefficients, then check whether they still fits observations from a different initial condition. On one system, a second trajectory distinguished two equations that fit the first equally well. Presentation also mattered. Across the tested systems, models recovered more equations on average from numerical samples than from plots of the same data (Sections 4.1 and 4.5).
About choosing a single equation:
A run counted as successful if the retained set contained the correct terms with accurate coefficients. The screen judged each proposal by its fit with the written coefficients and by the stability of coefficients fitted on different parts of the trajectory. A proposal with the correct terms could therefore lose to an alternative that fit better and had equally stable coefficients. Choosing just the equation with the best fit gave 13 to 66 correct choices per 100 runs across models, within two successes of the whole set for each model in its strongest configuration. Retaining alternatives exposed cases where one trajectory was consistent with several equations (Section 4.5, Tables 56–57).
About hints, derivative inputs and feedback:
Naming a missing term from the known equation would give the model part of the answer. We identified missing terms after the unaided runs by comparing the proposals with the known equations. We tested whether supplying derivative values helped. For data that varied across space, the observations and derivatives were sampled separately. Explicitly pairing them at the same locations remains untested. We also fitted coefficients to the terms the models proposed, on fresh solutions selected for accurate fitting. The fits improved recovery, but the effect of returning that feedback to the model for another attempt remains open (Table 1).