acceptodds
Under review as a conference paper at ICLR 2027

LLM-PySR: Large Language Model-Guided Symbolic Regression via Search Control

Abstract

Scientific equation discovery must combine broad domain priors with strict numerical testing. Symbolic regression supplies numerical grounding but faces a combinatorial search space, whereas many language-model systems ask the model to propose or select formulas directly.We systematically test how alternative divisions of decision-making authority between language models and numerical symbolic regression affect equation discovery. We compare role specifications in which the language model acts as equation author, candidate decider or search controller, alongside end-to-end language-model and purely numerical baselines. In the controller setting we propose here, implemented as LLM-PySR, language models specify variables, operators, transformations and search depth; symbolic regression enumerates and fits expressions; and fixed rules with numerical safeguards govern retention. Across 74 AI-Feynman equations and seven complex formula-recovery tasks, search control achieved the strongest observed balance of accuracy, complexity, stability and cost. Notably, on an independent battery dataset, LLM-PySR identified a compact piecewise-linear relation between early voltage-curve displacement and cycle life. Overall, LLM-PySR provides a token-efficient, interpretable, and empirically effective approach for scientific equation discovery, bridging the gap between LLM guidance and classical symbolic regression.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Symbolic regresion, LLM agent, ai for science, explainability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xiHHOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Recognizing the potential roles LLMs can play in symbolic regression, including providing semantic priors before the search, constructing expressions during the search, and selecting final models after the search, the authors developed LLM-PySR, in which the LLM primarily contributes to the first stage. Based on experimental results primarily on AI-Feynman benchmarks, the authors found that symbolic regression relies more on semantic priors that shape the space in which the numerical search is conducted, provided by an LLM, whereas the construction of expressions and the final selection of models should instead remain grounded in explicit numerical procedures.

Critical strengths

The authors categorize the role of LLMs in symbolic regression into three distinct parts (LLM as a search controller, LLM as an equation author, and LLM as a candidate decider) and validate the benefits of each through ablation studies. Results across 74 problems from the AI-Feynman benchmark suggest that employing an LLM as a search controller (i.e., LLM-PySR) yields the best numerical performance while requiring lower computational time and fewer tokens.

Critical weaknesses

Regarding method design: (W1) Beyond serving as a search controller, equation author, or candidate decider, the recently emerged paradigm of "agentic symbolic regression" (e.g., SR-Scientist[1], KeplerAgent[2], MOT-SR[3], SRHarness[4], and HarnessSR[4]) endows LLMs with a new capability: the ability to analyze data directly — either through code execution or by utilizing pre-designed tools — to gain immediate insights into the patterns hidden within the data. While this process bears some resemblance to the role of a search controller, it goes further: the LLM does not merely rely on the scientific context of the problem, but actively manipulates the data to extract insights. Moreover, this step does not necessarily occur at the beginning of the search; rather, it is often interleaved with other stages like the proposal or selection of candidate formulas. In fact, this role is quite important and may even be more important than the three roles considered by the authors, since the aforementioned work that utilizes this capability can achieve better performance on benchmarks that are much more complex than AI-Feynman. However, while the authors attempted to systematically assess the role of LLMs in symbolic regression, they overlooked this crucial point.

Regarding experimental results: (W2) The experiments conducted by the authors are insufficient. First (W2.1), the AI-Feynman benchmark consists of 100 problems and 19 bonus problems; however, the authors evaluated only 74 of these and claim that the remaining problems lacked a sufficient sample size (i.e., fewer than 500 samples). This is factually incorrect, as all 119 problems actually have a sample size of 10510^5. Secondly (W2.2), recent research in symbolic regression, particularly those leverage LLMs, has increasingly shifted toward the LLM-SRBenchmark, which features a broader and more challenging set of problems. Although the authors evaluated 7 problems from this benchmark, the benchmark actually comprises 111 LSR-Transform problems and 129 LSR-Synth problems. Therefore, an evaluation based on only 7 of these can hardly be considered "representative".

Apart from these, there are also some minor issues:

  • M1: In the caption for Table 2, the authors state that "the best wall-clock time for LLM-based methods is shown with underline"; however, Table 2 contains neither any underlined entries nor a column for wall-clock time.
  • M2: In Figure 3, the samples labeled as "Out-of-domain" (OOD) overlap with those labeled as "Train" (ranging from -0.06 to -0.02), which means that these so-called OOD samples are not truly out-of-domain.

Questions for authors

See the "Weaknesses" part.

2 comments
AuthorsOpenReview ↗

Response to reviewer 3

We thank the reviewer for your efforts and valuable suggestions.

  1. We agree that the recently developed agentic symbolic regression frameworks introduce an important direction beyond the conventional LLM-based equation-generation paradigm. We are also aware of these recent approaches, including SR-Scientist, KeplerAgent, MOT-SR, SRHarness, and HarnessSR. In fact, we have also observed in our own applications that using an agent harness such as Codex to further analyse and post-process the results generated by LLM-PySR can be quite effective. However, most of these works emerged after our submission, making it difficult to conduct a comprehensive comparison with all of them within the current review period.

We would also like to clarify that LLM-PySR already provides the LLM with numerical data samples for direct inspection. Specifically, the LLM is instructed to analyse internal relation patterns from the first 100 rows of the dataset. However, unlike the recently proposed agentic approaches, our framework does not explicitly provide code-execution or general-purpose tool-use capabilities for active data analysis.

Nevertheless, we believe that all these approaches are consistent with the central idea of our work: rather than asking LLMs to directly generate equation skeletons, it is more promising to let them act as agents that guide and interact with numerical symbolic regression tools. We will acknowledge this emerging direction and clarify the scope of our work in the revised manuscript.

  1. We apologize for an inaccurate statement in the original manuscript. The reason for excluding some AI-Feynman equations was not insufficient sample size. In fact, all 119 equations in the benchmark contain more than 500 samples. The actual reason is that our initial feature-fusion step requires at least two input variables. Therefore, we excluded equations containing only a single input variable, leaving 74 equations for evaluation. We will correct this explanation in the revised manuscript.

  2. We apologize for not evaluating all formulas in LLM-SRBench. Our original intention was to follow the experimental settings of DrSR, which selected only these seven formulas. We have now conducted additional experiments on full LLM-SRBench. The results are shown below:

AlgorithmACC_0.1 mean ↑NMSE mean ↓Tokens mean ↓Time mean (s) ↓Nodes mean ↓Structural similarity ↑Symbolic exact rate ↑
LLM-PySR (new)76.1%3.064E-02121,824.71,669.912.570.45590.0%
LLM-SR95.0%9.163E-013,670,572.746,258.252.300.36560.0%
DrSR58.0%4.436E+035,723,128.01,601.737.560.43190.0%
PySR70.7%1.885E-010.0336.724.530.41190.0%

These results show that although LLM-SR achieves the highest ACC, its NMSE is not better than that of LLM-PySR. Meanwhile, LLM-PySR achieves the lowest expression complexity and the highest structural similarity among all methods, although none of the methods successfully recovers the complete formulas.

Thank you for the response and the additional experiments. Regarding your first response, three of the five works were publicly available before this paper was submitted to ICLR 2027 (SR-Scientist in October 2025, KeplerAgent in February 2026, and MOT-SR in July 2026), so the claim that “most of these works emerged after our submission” is incorrect. Moreover, given your statement that “all these approaches are consistent with the central idea of our work,” a systematic comparison becomes particularly necessary.

More importantly, your third response reports a 0% symbolic recovery rate for LLM-PySR on the LLM-SRBench. However, SR-Scientist reports a symbolic accuracy of 7.75% on the LSR-Synth split, and KeplerAgent reports 35.14% on the LSR-Transform split. Although these figures may have been derived under different evaluation schemes, they highlights the necessity of direct comparison to clarify the actual effectiveness of LLM-PySR in recovering symbolic expressions.

Reviewer_GkBJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LLM-PySR, a symbolic regression framework that leverages LLMs to guide search-space configuration, including variable selection, operator selection, feature engineering, and search depth, while relying on PySR for expression search and constant fitting. The authors systematically investigate three LLM roles—search controller, equation author, and candidate decider—and compare their performance against existing symbolic regression methods. Experiments on 74 AI-Feynman equations, seven complex formula-recovery tasks, and two real-world scientific datasets show that LLM-guided search control achieves a favorable balance among predictive accuracy, expression complexity, and computational cost.

Critical strengths
  1. Systematic investigation of LLM roles in symbolic regression. The paper compares three distinct LLM roles—search controller, equation author, and candidate decider—within a unified framework. This provides useful empirical insights into how different allocations of decision-making authority affect symbolic regression performance.

  2. Extensive experimental evaluation. The paper evaluates the proposed framework on 74 AI-Feynman equations, seven complex formula-recovery tasks, and two real-world scientific datasets. The results demonstrate competitive predictive accuracy and improved computational efficiency over several existing LLM-based symbolic regression methods, supporting the practical value of LLM-guided search control.

Critical weaknesses
  1. Limited methodological novelty. Using LLMs to guide PySR's search configuration has already been explored by LangLaw. LLM-PySR mainly extends this idea with additional search controls (operator selection, feature engineering, and custom operators), and these extensions appear incremental. The paper also lacks component-level ablations that would show each extension's individual contribution.

  2. Marginal gains over existing methods. On AI-Feynman, LLM-PySR improves only slightly on LangLaw in ACC (0.958 vs. 0.956), and LangLaw achieves lower NMSE and token cost. Without repeated runs and statistical significance tests, it is unclear whether these differences are meaningful. Comparisons under matched computational budgets would also better support the claimed efficiency advantages.

  3. Inconsistent data partitioning in the battery experiment. Section 4.3.1 reports 41 training samples, 43 validation samples, and 40 independent test samples, whereas Figure 3 shows 20 training, 21 validation, 43 ID test, and 40 OOD test samples. Please provide a more detailed explanation of this discrepancy and clarify how each subset was used during training, model selection, and evaluation.

Questions for authors

Q1. The primary concerns in Weaknesses 1–3.

Q2. In Section 4.3.2, the authors claim that all competing methods introduce additional variables, whereas Table 4 shows that LangLaw also uses only one variable.

Q3. The computation of the symbolic exact recovery rates in Table 2 also requires clarification, as the reported percentages do not directly correspond to integer recovery counts over 74 equations.

Q4. Minor issues include the incorrect caption of Table 2, the misspelling of "Structural Similarity," inconsistent notation, and several grammatical and formatting errors.

2 comments
AuthorsOpenReview ↗

Response to reviewer 2

We thank the reviewer for your efforts and valuable suggestions.

  1. We agree that simply combining an LLM with PySR is not entirely novel. However, our main contribution is not merely this combination, but the investigation of how decision-making responsibilities should be allocated between LLMs and numerical symbolic regression. Specifically, we show that using an LLM to configure and constrain the search space, while leaving expression generation and exploration to a numerical symbolic regression engine, is more effective than directly asking the LLM to generate candidate equation structures. We systematically investigate this design through three controlled roles: search controller, equation author, and candidate decider. Recent approaches, such as SR Harness, also explore providing symbolic regression tools to LLM agents, reflecting growing interest in this tool-assisted paradigm.

  2. The similar performance of LLM-PySR and LangLaw is itself informative. Both methods outperform the direct equation-generation approaches in terms of their overall balance between predictive performance and computational cost. This supports our central observation that assigning the LLM the role of search controller is a promising alternative to directly generating equation skeletons. LLM-PySR further provides a broader search-control interface, resulting in lower expression complexity, higher structural similarity, and shorter runtime than LangLaw in our experiments.

  3. We apologize for the unclear description of the data partitioning. The original dataset provides three subsets: 41 training cells, 43 validation cells, and 40 independent test cells. We retained this original partitioning. However, LLM-PySR further splits the original training set internally into two subsets: 20 cells for PySR training and 21 cells for internal validation, which is used to evaluate PySR candidates and provide feedback during the iterative search. The original 43-cell validation set and 40-cell independent test set are retained as separate external evaluation subsets. Therefore, Figure 3 shows the internal 20/21 split of the original training set, together with the two external subsets of 43 and 40 cells. We will clarify this distinction in the revised manuscript and correct the corresponding descriptions and figure labels.

Thank you for your detailed response. I am inclined to keep the original score.

Reviewer_QhjhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LLM-PySR, where an LLM configures variables, operators, features, and search depth while PySR searches in the space of mathematical equations. It compares using the LLM as a search controller, equation candidate proposer, or candidate selector. The approach shows useful accuracy–cost trade-offs compared to LLM-only frameworks like LLM-SR, but the novelty compared to earlier hybrid LLM+search works and evidence for preferring this division of roles need clearer support and direct experimental comparisons.

Critical strengths
  • I think the question that this paper raise on asking “which decision does the LLM own?” with matched ablation experiments is a clean and practically helpful question in using LLMs for model/equation discovery.

  • The method empirically shows good gains in the reduction of token usage relative to LLM-SR while maintaining strong predictive accuracy.

  • I agree with author notes that reporting expression complexity and exact recovery alongside prediction error helps assess whether the method actually discovers meaningful equations.

Critical weaknesses
  • Prior methods in LLM-based equation discovery LaSR [1] and LangLaw [2] already combine LLM guidance with PySR-based evolutionary search in a hybrid framework. Here, the LLM configures searches rather than guiding mutations, but how much does that distinction help in practice? A direct comparison with these methods under comparable budgets sand same LLM backbones is necessary.

  • Standalone PySR runs for about 8 seconds versus 493 seconds for LLM-PySR. How much of the improvement remains with equal compute or additional PySR restarts? Performance-versus-budget curves for fair analysis are needed. Also no seeds or significance tests are reported in the results.

  • Could the authors evaluate the full LLM-SRBench and report repeated-seed uncertainty? Variation across equations does not establish run-to-run stability and I think the scale of experiments now are not enough to verify the claims made by authors in the paper.

  • Do results hold with anonymized variable names, to rule out memorization challenges of using LLMs for these known problems?

  • Appendix D shows that LLM reviewer scores still affect candidate retention and incumbent updates. Could the authors include a controller variant with fully numerical selection to support the central claim?

Minor Comment: Section 4.3 describes 41/43/40 train/validation/test cells, while Figure 3 labels 20/21/43/40. Which split produced the reported results, and was the 43-cell set used for selection or independent testing?

[1] Symbolic Regression with a Learned Concept Library, NeurIPS, 2024

[2] Discovery of Interpretable Physical Laws in Materials via Language-Model-Guided Symbolic Regression, 2026

Questions for authors

check section above

2 comments
AuthorsOpenReview ↗

Response to reviewer 1

We thank the reviewer for the positive assessment and these helpful suggestions.

  1. We agree that combining LLMs with evolutionary algorithms such as PySR has already been explored in prior methods, including LaSR and LangLaw. In our experiments, we re-evaluated LangLaw. Overall, LangLaw achieves performance relatively close to LLM-PySR in terms of ACC₀.₁, NMSE, node complexity, and structural similarity, and performs relatively well among all evaluated methods. This suggests that using a tool such as PySR for symbolic regression, rather than directly asking an LLM to recommend candidate skeletons, is indeed a promising strategy. When evaluating LaSR, however, we repeatedly encountered the error "LaSR produced no usable concepts", which prevented us from completing the experiments. Therefore, we did not include LaSR results in the paper.

  2. We appreciate this reasonable suggestion. We conducted additional experiments with larger computational budgets for PySR. We set the maximum number of iterations to 100K and controlled the runtime to allow PySR to stop early, with two time limits of 300 s and 600 s, respectively. The results are shown below:

ConfigurationACC_0.1 ↑NMSE ↓Node complexity ↓Structural similarity ↑Symbolic exact ↑
PySR 300 s99.7568%9.2307 × 10²⁰¹25.86490.38912.7027% (2/74)
PySR 600 s98.5000%+∞26.67570.37232.7027% (2/74)
PySR 7 s92.2%0.1020.180.47404.2%

Both longer-budget settings produced extreme predictions on equation III.15.12. In the 300 s setting, its NMSE was 6.8307 × 10²⁰³, which dominated the overall mean.

In addition, the original test evaluation of equation I.29.16 in the 600 s setting resulted in an error. After recomputing the predictions using the saved formula, we found that 13 out of 1,500 test points produced non-finite predictions. According to the original evaluation protocol, this equation was assigned ACC = 0 and NMSE = +∞. After excluding these two cases, the results are as follows:

ConfigurationACC_0.1 ↑NMSE ↓Node complexity ↓Structural similarity ↑Symbolic exact ↑
PySR 300 s99.8519%9.5894 × 10⁻⁵25.69440.39082.7778% (2/72)
PySR 600 s99.8491%5.4233 × 10⁻⁵26.50000.37362.7778% (2/72)
  1. We apologize for not evaluating all formulas in LLM-SRBench. Our original intention was to follow the experimental settings of DrSR, which selected only these seven formulas. We have now conducted additional experiments on LLM-SRBench. The results are shown below:
AlgorithmACC_0.1 mean ↑NMSE mean ↓Tokens mean ↓Time mean (s) ↓Nodes mean ↓Structural similarity ↑Symbolic exact rate ↑
LLM-PySR (new)76.1%3.064E-02121,824.71,669.912.570.45590.0%
LLM-SR95.0%9.163E-013,670,572.746,258.252.300.36560.0%
DrSR58.0%4.436E+035,723,128.01,601.737.560.43190.0%
PySR70.7%1.885E-010.0336.724.530.41190.0%

These results show that although LLM-SR achieves the highest ACC, its NMSE is not better than that of LLM-PySR. Meanwhile, LLM-PySR achieves the lowest expression complexity and the highest structural similarity among all methods, although none of the methods successfully recovers the complete formulas.

We also apologize for not conducting repeated-seed experiments across all results. This was largely constrained by the time required to complete multiple full runs of all algorithms within the available period. We will supplement the experiments as much as possible and update the results accordingly.

  1. In LLM-PySR, the variable identifiers used in symbolic regression are fully anonymized, with the algorithm assigning new names to each variable. However, the variable descriptions and units are still provided to the algorithm as context. For example:

"sample_rows": [ { "x1": 0.2144957333803177, "x2": 29.2076416015625, "y": 96.47188568115234 } ]

  1. We agree that LLM reviewer scores can affect the search strategy. We therefore tried a relatively simple strategy: selecting the formula with the highest ACC among all candidates generated over the 10 LLM-PySR iterations. The results are actually better than those of the original LLM-PySR:

ACC: 0.989 (0.958)

NMSE: 26.709 (0.034), because of an outlier I.9.18, if discarded 6.876*10^-7

Node complexity: 14.81 (15.05)

Structural similarity: 0.724 (0.610)

Symbolic exact: 0.239 (0.113)

It is actually better to use a pure numerical selection, supporting our main claim of using LLM only as controller.

Reviewer_QhjhOpenReview ↗

Official Comment by Reviewer Qhjh

Thank you for responding to my comments and for new experiments. I will keep my positive rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.