- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Recognizing the potential roles LLMs can play in symbolic regression, including providing semantic priors before the search, constructing expressions during the search, and selecting final models after the search, the authors developed LLM-PySR, in which the LLM primarily contributes to the first stage. Based on experimental results primarily on AI-Feynman benchmarks, the authors found that symbolic regression relies more on semantic priors that shape the space in which the numerical search is conducted, provided by an LLM, whereas the construction of expressions and the final selection of models should instead remain grounded in explicit numerical procedures.
Critical strengths
The authors categorize the role of LLMs in symbolic regression into three distinct parts (LLM as a search controller, LLM as an equation author, and LLM as a candidate decider) and validate the benefits of each through ablation studies. Results across 74 problems from the AI-Feynman benchmark suggest that employing an LLM as a search controller (i.e., LLM-PySR) yields the best numerical performance while requiring lower computational time and fewer tokens.
Critical weaknesses
Regarding method design: (W1) Beyond serving as a search controller, equation author, or candidate decider, the recently emerged paradigm of "agentic symbolic regression" (e.g., SR-Scientist[1], KeplerAgent[2], MOT-SR[3], SRHarness[4], and HarnessSR[4]) endows LLMs with a new capability: the ability to analyze data directly — either through code execution or by utilizing pre-designed tools — to gain immediate insights into the patterns hidden within the data. While this process bears some resemblance to the role of a search controller, it goes further: the LLM does not merely rely on the scientific context of the problem, but actively manipulates the data to extract insights. Moreover, this step does not necessarily occur at the beginning of the search; rather, it is often interleaved with other stages like the proposal or selection of candidate formulas. In fact, this role is quite important and may even be more important than the three roles considered by the authors, since the aforementioned work that utilizes this capability can achieve better performance on benchmarks that are much more complex than AI-Feynman. However, while the authors attempted to systematically assess the role of LLMs in symbolic regression, they overlooked this crucial point.
Regarding experimental results: (W2) The experiments conducted by the authors are insufficient. First (W2.1), the AI-Feynman benchmark consists of 100 problems and 19 bonus problems; however, the authors evaluated only 74 of these and claim that the remaining problems lacked a sufficient sample size (i.e., fewer than 500 samples). This is factually incorrect, as all 119 problems actually have a sample size of . Secondly (W2.2), recent research in symbolic regression, particularly those leverage LLMs, has increasingly shifted toward the LLM-SRBenchmark, which features a broader and more challenging set of problems. Although the authors evaluated 7 problems from this benchmark, the benchmark actually comprises 111 LSR-Transform problems and 129 LSR-Synth problems. Therefore, an evaluation based on only 7 of these can hardly be considered "representative".
Apart from these, there are also some minor issues:
- M1: In the caption for Table 2, the authors state that "the best wall-clock time for LLM-based methods is shown with underline"; however, Table 2 contains neither any underlined entries nor a column for wall-clock time.
- M2: In Figure 3, the samples labeled as "Out-of-domain" (OOD) overlap with those labeled as "Train" (ranging from -0.06 to -0.02), which means that these so-called OOD samples are not truly out-of-domain.
Questions for authors
See the "Weaknesses" part.
2 comments
Thank you for the response and the additional experiments. Regarding your first response, three of the five works were publicly available before this paper was submitted to ICLR 2027 (SR-Scientist in October 2025, KeplerAgent in February 2026, and MOT-SR in July 2026), so the claim that “most of these works emerged after our submission” is incorrect. Moreover, given your statement that “all these approaches are consistent with the central idea of our work,” a systematic comparison becomes particularly necessary.
More importantly, your third response reports a 0% symbolic recovery rate for LLM-PySR on the LLM-SRBench. However, SR-Scientist reports a symbolic accuracy of 7.75% on the LSR-Synth split, and KeplerAgent reports 35.14% on the LSR-Transform split. Although these figures may have been derived under different evaluation schemes, they highlights the necessity of direct comparison to clarify the actual effectiveness of LLM-PySR in recovering symbolic expressions.
Response to reviewer 3
We thank the reviewer for your efforts and valuable suggestions.
We would also like to clarify that LLM-PySR already provides the LLM with numerical data samples for direct inspection. Specifically, the LLM is instructed to analyse internal relation patterns from the first 100 rows of the dataset. However, unlike the recently proposed agentic approaches, our framework does not explicitly provide code-execution or general-purpose tool-use capabilities for active data analysis.
Nevertheless, we believe that all these approaches are consistent with the central idea of our work: rather than asking LLMs to directly generate equation skeletons, it is more promising to let them act as agents that guide and interact with numerical symbolic regression tools. We will acknowledge this emerging direction and clarify the scope of our work in the revised manuscript.
We apologize for an inaccurate statement in the original manuscript. The reason for excluding some AI-Feynman equations was not insufficient sample size. In fact, all 119 equations in the benchmark contain more than 500 samples. The actual reason is that our initial feature-fusion step requires at least two input variables. Therefore, we excluded equations containing only a single input variable, leaving 74 equations for evaluation. We will correct this explanation in the revised manuscript.
We apologize for not evaluating all formulas in LLM-SRBench. Our original intention was to follow the experimental settings of DrSR, which selected only these seven formulas. We have now conducted additional experiments on full LLM-SRBench. The results are shown below:
These results show that although LLM-SR achieves the highest ACC, its NMSE is not better than that of LLM-PySR. Meanwhile, LLM-PySR achieves the lowest expression complexity and the highest structural similarity among all methods, although none of the methods successfully recovers the complete formulas.