acceptodds
Under review as a conference paper at ICLR 2027

EvoMO-SR: Multiobjective LLM-based Evolution of Symbolic Expressions with substructure guidance

Abstract

Symbolic Regression (SR) is a data-driven method for scientific discovery which searches for interpretable analytical relationships within data. Recently, Large Language Models (LLMs) have also had a significant impact on scientific discovery, enabling the automation of various stages of the process. For these reasons, the possibility of harnessing the embedded scientific knowledge and programming capabilities of LLMs to solve SR tasks has emerged, showing promising performance compared with traditional methods. We propose EvoMO-SR, a novel LLM-driven SR framework in which the LLM generates equation skeletons, with their coefficients fitted separately by an external optimizer. The framework includes a multi-objective survival selection which controls bloating by balancing accuracy and complexity, and a substructure guidance mechanism which mutates expressions with candidate reusable building blocks. EvoMO-SR achieves the best accuracy in seven of the eight in-domain and out-of-domain settings for LSR-Synth, using a small LLM model, i.e., Llama-3.1-8B-Instruct. We also evaluated structural recovery through two symbolic accuracy metrics based on canonicalized subtree overlap and term matching, showing that our method has a greater probability of recovering highly accurate symbolic structures.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Symbolic Regression, Large Language Models, evolutionary computation, LLM-driven Symbolic Regression

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fpEEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces EvoMO‑SR, an LLM‑driven symbolic regression (SR) framework. The method uses a Large Language Model (Llama‑3.1‑8B‑Instruct) to generate equation skeletons, which are then fitted by an external optimizer (L‑BFGS‑B). The framework incorporates a multi‑objective survival selection mechanism to balance accuracy (NMSE) and complexity (AST node count), and a substructure guidance module that extracts additive subtrees from evolved expressions and reuses them in subsequent mutations. Experiments on the LSR‑Synth benchmark and two physics‑based datasets (oscillator1/2) demonstrate that EvoMO‑SR outperforms existing LLM‑based and traditional SR baselines in seven out of eight domain‑split comparisons.

Critical strengths
  1. The paper tackles an important and timely problem: leveraging LLMs for interpretable scientific discovery via symbolic regression.

  2. The proposed substructure guidance idea is intuitive and could potentially help mitigate bloating and improve search efficiency.

  3. Empirical results show promising numerical accuracy gains, especially on out‑of‑distribution (OOD) splits, using a relatively small LLM backbone.

Critical weaknesses
  1. The manuscript suffers from overly long paragraphs with unclear transitions between ideas, making the narrative difficult to follow. For instance, the Related Work section is presented as a single, unbroken block of text, which severely hampers readability and fails to group related methods logically.

  2. The paper positions itself as a novel combination of LLM generation, GP‑style evolution, and substructure extraction. However, the distinction from prior LLM‑based SR works—particularly LLM‑SR (Shojaee et al., 2025a) and LaSR (Grayeli et al., 2025)—is not clearly articulated. It is unclear what fundamental limitation of existing approaches is addressed, or what theoretical benefit the added modules bring. The authors state that EvoMO‑SR “enhances GP search with LLM‑generated concepts,” but LaSR already does this; the essential difference between the two methods is never explained. The current presentation gives the impression of a straightforward fusion of existing techniques rather than a principled advancement.

  3. The first sentence of the Related Work states: “SR is a supervised machine learning method that aims to retrieve the mathematical expression that best fits a given dataset.” This characterization is debatable. While deep‑learning‑based SR methods may use synthetic expressions as supervision, many search‑based algorithms (e.g., GP, evolutionary strategies) rely solely on a fitness function (e.g., MSE) without requiring ground‑truth expression labels. Labelling SR universally as “supervised” is therefore inaccurate and potentially misleading.

  4. Table 3 reports median NMSE and symbolic accuracy (SA) values across runs. In symbolic regression, the primary goal is to discover at least one expression that is both sufficiently accurate and parsimonious. Averaging over all runs via the median obscures whether the method successfully found such an expression in individual trials. Reporting best‑run performance or success rates (e.g., fraction of runs recovering a ground‑truth subtree) would be more informative.

  5. The ablation experiment in Table 4 only compares performance with/without substructure guidance on two specific datasets (oscillator1 and oscillator2). This is inadequate to establish the general utility of the proposed components. A thorough ablation should: (1) Isolate the contribution of each major module: (a) LLM‑only prediction (no evolution), (b) GP‑only search (no LLM), (c) full model without substructure guidance. (2) Evaluate on a substantially larger and more diverse set of expressions (beyond just two cases) to guard against dataset‑specific overfitting. Without such experiments, it is difficult to attribute performance gains to the proposed fusion rather than chance.

Questions for authors
  1. Novelty Clarification: Could you explicitly delineate the core differences between EvoMO‑SR and LaSR/LLM‑SR? What specific weakness of those methods does your framework overcome, and how does the multi‑objective survival selection fundamentally improve upon their sampling‑based or single‑objective strategies?

  2. Theoretical Justification: Beyond empirical results on two datasets, is there any theoretical analysis or intuition suggesting that substructure guidance should accelerate convergence or improve symbolic recovery? How are substructures scored independently of the full expression’s fitness?

  3. SR as Supervised Learning: Do you agree that many SR algorithms (especially evolutionary ones) do not use expression labels and therefore should not be classified as supervised? Would you consider revising this statement to avoid confusion?

  4. Evaluation Protocol: Why was the median chosen as the summary metric in Table 3? Would it be possible to report the number of successful recoveries (e.g., SA > threshold) or the best‑of‑run performance to better reflect SR’s goal of finding a single high‑quality expression?

  5. Expanded Ablation: Could you provide additional ablation results that remove the LLM or the GP loop entirely, and test on a wider range of problems (e.g., from LSR‑Synth or randomly generated functions) to demonstrate that the improvements are not limited to the two oscillator datasets?

1 comment
AuthorsOpenReview ↗

We clarify novelty-related concerns; we show additional experiments for substructures; we clarify SR definition; we clarify the evaluation protocol and provide best expressions for randomly generated functions; we provide an additional ablation study.

We thank the reviewer for the constructive feedback and for recognizing the importance of the problem.

  1. Novelty Clarification: More detailed qualitative comparisons are available in Appendix E. EvoMO-SR introduces an explicit population-based evolutionary loop. While LaSR performs SR with an evolutionary loop as well, its search engine is managed by PySR. LLM-SR uses some evolutionary-related components, but its loop mainly evolves expressions based on a multi-island sampling approach. The overall framework includes the possibility to customize other evolutionary-related parameters (as described in Appendix A.1). Concerning multi-objective optimization, EvoMO-SR exploits numerical accuracy and expression complexity to determine population survival during the evolutionary loop. The ablations in Appendix C.3 (Tables 5 and 6) show that this consistently reduces expression complexity. Concerning the objection raised by the reviewer, we would like to clarify that the sentence "enhances GP search with LLM‑generated concepts" actually refers to LaSR in our manuscript (see Section 2). In fact, our method does not use LLM-generated concepts to enhance the search. This is what distinguishes us from LaSR in what we use to guide LLM-based search.

  2. Theoretical Justification: The intuition in integrating substructure guidance is that mathematical pieces discovered by the LLM during the evolution may remain useful when modifying other candidates. By retaining such substructures in an archive could be a way in which the search can reuse potentially valuable symbolic information. For the concerns raised, we carried out additional experiments on oscillator2, CRK34 problem (LSR-Synth), and the P3 problem (randomly generated function), each for 5 runs. We investigated whether reused substructures were beneficial by removing the added substructure for each incorporation case. Results shown that removing the substructure increased ID NMSE in 72.7% of cases and OOD NMSE in 52.1% of cases (higher is worse). Additionally, for each substructure present in the ground truth, we recorded their improvement rate (on parent expressions) and their presence in the final expression. For example for oscillator2, sin(t)sin(t) improved parents in 72.2% of cases and was present in 5/5 final expressions; for CRK34 A2A^2 the improvement was in 57.6% of parents, presence 5/5; for P3 x02x_0^2 the improvement was in 70% of parents, presence 4/5

Based on the results obtained, we aim to show the value of introducing the substructure guidance mechanism, without, however, claiming that it is, a method that consistently improves performance.

  1. SR as Supervised Learning: we agree that many SR algorithms are tested over datasets that do not have ground truth expressions. However, there is a distinction with the actual label of the SR datasets. A ground truth is not a label associated with an input. Dataset’s input-output pairs in SR datasets are numerical only, since SR is a regression algorithm. In fact, SR methods can solve standard regression problems (as seen in SRBench - La Cava, William, et al., 2021). Since the SR outputs a symbolic expression, it is surely compared against the ground truth expression when available, but it does not constitute the label. When the ground truth is not available, the SR output (i.e., the symbolic expression) is evaluated by computing the predicted y, and compared against the real y (the target value in the dataset). For these reasons, the SR is recognized in literature as a supervised algorithm.

  2. Evaluation Protocol: taking the median into account ensures that problems where these methods fail do not have too large an impact on overall performance. For P1, the best expression (based on the lowest NMSE) is obtained in 3 out of 5 runs, where the retrieved expression is the same and is equal to the ground truth. For P2, the best expression found was exactly equal to the ground truth expression. For P3, the best expression has SA_Term = 0.33 and is 1.51–1.95x02+0.33x12+2.36x22+1.37cos(x1)+6.17×10−7x01.51 – 1.95x_0^2 + 0.33x_1^2 + 2.36x_2^2 + 1.37cos(x_1) + 6.17 \times 10^-7x_0. For P4, the best expression has SA_Term = 0.4 and is −2.10–2.16sin(x0)–1.08cos(x0)+1.9x12+2.38x1x2−2.23cos(x2)+0.0001x22−0.0002x1+0.000004x2+0.0002sin(x1)-2.10 – 2.16sin(x_0) – 1.08cos(x_0) + 1.9x_1^2 + 2.38x_1x_2 -2.23cos(x_2) + 0.0001x_2^2 -0.0002x_1 + 0.000004x_2 + 0.0002sin(x_1). All the substructures in the ground truth are present in the final expression, with some spurious terms with low coefficients.

  3. Expanded Ablation: We acknowledge that performing more ablation studies could be beneficial. However, we would like to clarify that in Appendix C.3.1 (Figure 13) we compared results obtained by using GP-only mutation and LLM-guided mutation. Following your suggestion, we carried out ablations on randomly generated functions (P1 and P4) with (wg) and without guidance (wog).

  • P1: median NMSE ID wg is 2.56e-32 while wog is 1.06e-14, NMSE OOD wg is 1.40e-32 while wg is 8.09e-14
  • P4: NMSE ID wg is 3.63e-6 while wog is 2.08e-5, NMSE OOD wg is 1.66e-3 while wo is 9.13e-3.
Reviewer_zf41OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes EvoMO-SR, an LLM-based symbolic regression framework that (1) employs Pareto-based multi-objective selection to balance predictive accuracy and expression complexity, and (2) introduces a substructure archive to accumulate and reuse knowledge during evolution. The paper evaluates EvoMO-SR on LSR-Synth from LLM-SRBench, demonstrating promising performance in both in-domain (ID) and out-of-domain (OOD) settings, along with additional ablation studies.

Critical strengths

1, Comprehensive evaluation with promising performance. The paper provides a comprehensive evaluation on all 129 problems in LSR-Synth from LLM-SRBench, with five independent runs. The results demonstrate promising performance, particularly in OOD settings.

2, Pareto-based survival selection. The paper addresses the important issue of expression bloating in symbolic regression by incorporating Pareto-based survival selection to balance predictive accuracy and expression complexity.

Critical weaknesses

1, Insufficient differentiation from recent related works. The methodological novelty could be better established through a more detailed comparison with recent works. The accumulation and reuse of executable mathematical structures have already been explored in recent approaches. Additionally, existing symbolic regression methods already employ complexity regularization to discourage overly complex expressions. The authors should clarify how the proposed Pareto-based multi-objective selection differs from these approaches and what additional benefits it provides.

2, Limited generality of additive substructure extraction. The proposed substructure-guidance mechanism focuses on newly introduced additive terms identified through parent–offspring comparisons. However, the approach does not explicitly identify reusable subexpressions at arbitrary levels of the expression tree. For instance, in (f(x)=\exp(\sin(x^2))), the subexpression (\sin(x^2)) is a potentially reusable building block, but it is nested within an exponential function rather than appearing as an independent additive term. It remains unclear whether the proposed mechanism can effectively capture and reuse such complex structures.

3, Insufficient ablation studies and hyperparameter sensitivity analysis. The complete component ablation is conducted only on two oscillator problems, with mixed results. Additionally, several important hyperparameters are fixed without sensitivity analysis, leaving the robustness of the framework to these choices unclear.

4, Presentation and clarity issues. The presentation could be improved in several aspects. In particular, Figure 1 is overly complex and difficult to follow, making it challenging to understand the overall workflow and interactions between different components. Moreover, the methodology section is relatively compact. For example, the criterion for selecting the final expression from the Pareto front is not clearly explained.

Questions for authors

See weaknesses.

2 comments
AuthorsOpenReview ↗

We indicate differences with related works; we clarify the limited generality issue; we acknowledge limitations in ablation studies and provide an additional experiment.

We thank the reviewer for the careful assessment of our work and for recognizing the evaluation and promising performance. We would like to provide some clarifications concerning the raised weaknesses.

  1. Insufficient differentiation from recent related works: we aknowledge that multi-objective optimization, complexity regularization and the reuse of information acquired during search are not individually novel concepts in Symbolic Regression (SR). Our contribution is intended to introduce a specific LLM-driven evolutionary framework that combines explicit multi-objective survival selection with the extraction and reuse of mathematical substructures identified through parent-offspring comparisons. We would also like to highlight that the differences between EvoMO-SR and LLM-SR and LaSR are discussed briefly in Section 2 and in a more comprehensive way in Appendix E. LLM-SR maintains an experience buffer containing complete candidate expressions, which are sampled to enrich subsequent prompts. Although this mechanism incorporates evolutionary principles, this does not employ explicit parent-offspring survival selection within a population-based evolutionary algorithm. On the other hand, LaSR is built upon PySR and combines GP operators with LLM-based variation operators. The search is guided using a library of natural language concepts, while EvoMO-SR maintains an archive of reusable mathematical substructures. Concerning multi-objective optimization, our contribution in the LLM-based SR methods field is the explicit insertion of a Pareto-based selection within the evolutionary process. EvoMO-SR directly uses NMSE and expression complexity as competing objectives in population survival selection, through non-dominated sorting and crowding distance. Therefore, while the individual underlying techniques are established in traditional SR algorithms, we believe that their integration in LLM-based methods constitutes a novel methodological contribution.

  2. Limited generality of additive substructure extraction: we agree that the current substructure extraction mechanism focuses on newly introduced additive components rather than arbitrary subtrees at any depth of an expression tree. However, we would like to clarify that this restriction does not necessarily prevent the extraction of mathematically complex or nested expressions. Considering the expression f(x)=exp⁡(sin⁡(x2))f(x)=\exp(\sin(x^2)), if the LLM introduces a new additive component, the entire term exp⁡(sin⁡(x2))\exp(\sin(x^2)) can be identified as a candidate substructure. What the current mechanism does not explicitly support is the independent extraction of the nested component sin⁡(x2)\sin(x^2) from within the exponential term. However, we acknowledge that extending the substructure extraction process could increase the range of reusable mathematical pieces and represents a promising direction for future developments.

  3. Insufficient ablation studies and hyperparameter sensitivity analysis: we agree that additional experiments over a wider range of problems would further strengthen the analysis of the proposed components. However, we would like to clarify that the manuscript already includes several complementary ablation experiments beyond the comparison presented in Table 4. Appendix C.3 reports an ablation study in which substructure guidance and multi-objective survival selection are enabled or disabled. Appendix C.3.1 presents an additional experiment comparing LLM-driven mutation against conventional GP mutation and a hybrid strategy alternating between the two. Finally, Table 4 evaluates the effect of substructure guidance across three different LLM backbones, including Llama-3.1-8B-Instruct, Deepseek-v4-Flash, and GPT-5.4-nano. However, we acknowledge that these component-level ablations are limited to the two oscillator datasets, although extending the factorial ablation to the entire benchmark would entail a substantial additional computational cost. We carried out ablations on randomly generated functions (P1 and P4). On P1, NMSE ID with guidance is 2.56e-32 while without guidance is 1.06e-14, NMSE OOD with guidance is 1.40e-32 while without guidance is 8.09e-14. On P4, NMSE ID with guidance is 3.63e-6 while without guidance is 2.08e-5, NMSE OOD with guidance is 1.66e-3 while without guidance is 9.13e-3. So substructure guidance results in better performance for NMSE ID/OOD on both problems, although with small margin for P4.

  4. Presentation and clarity issues: we agree that the figure could be better explained and better coupled with the methodological section. We can extend the description for sure, adding more details of the interactions among the components with clear reference to the figure. Concerning the criterion used to select the final expression, we would like to clarify that it is described in Appendix D, where we state that the final expression is the one with the lowest training NMSE from the final Pareto front.

Thank you for the detailed response. The authors discussed the difference between EvoMo-SR and LLMSR/LASR, but some more recent works remain undiscussed (e.g., DrSR(Wang et al 2025)/SR-Scientist(Xia et al 2025). Additionally, my concerns regarding hyperparameter sensitivity, additive substructure, and presentation remain. Therefore, I will keep my score.

Reviewer_BLr9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EvoMO-SR, an LLM-based symbolic regression framework that combines LLM-driven equation generation and mutation, external coefficient optimization, multi-objective survival selection, and a reusable substructure archive. The method uses L-BFGS-B to fit numerical coefficients, balances predictive accuracy and expression complexity through Pareto-based selection, and reuses substructures extracted from promising offspring to guide subsequent mutations.

The authors evaluate EvoMO-SR on the 129-problem LSR-Synth benchmark, two oscillator datasets, and four randomly generated function datasets. Comparisons include LLM-based methods such as LLM-SR and LaSR, as well as several conventional symbolic regression baselines. The paper also reports repeated runs, ablation studies, statistical tests, and symbolic accuracy metrics based on AST subtree overlap and additive-term matching. The results indicate competitive numerical performance, particularly on OOD splits, although symbolic recovery remains limited and the benefits of substructure guidance are not consistent across all settings.

Critical strengths
  1. Broad empirical evaluation and substantial experimental effort.The authors evaluate EvoMO-SR across multiple scientific domains and randomly generated functions, compare it with several LLM-based and conventional symbolic regression baselines, and conduct repeated runs, ablation studies, and statistical tests. These extensive experiments provide a comprehensive evaluation of the proposed method across diverse datasets, model configurations, and evaluation settings.

2.A practical integration of evolutionary search and LLM-based equation generation.The framework combines explicit population-level survival selection with LLM-driven structural variation and a reusable substructure archive. Its Pareto-based selection mechanism addresses the trade-off between numerical accuracy and expression complexity, while the archive offers a practical way to reuse mathematical components discovered during search. The reported results show promising numerical performance, particularly on several OOD settings.

Critical weaknesses
  1. Unreliable Substructure Evaluation The substructure archive is a central distinguishing component of EvoMO-SR, whereas multi-objective selection and external coefficient optimization are established techniques in symbolic regression and evolutionary optimization. However, the archive evaluates a substructure using the NMSE of the complete offspring expression in which it appears, rather than measuring the substructure's individual contribution. This can lead to misleading credit assignment when a single LLM mutation introduces, removes, or modifies multiple terms. For example, consider a parent expression (f_{\mathrm{parent}}=x_1) and an offspring (f_{\mathrm{offspring}}=x_1+x_2^2+100x_3). Even if (x_2^2) is useful, the large (100x_3) term may substantially worsen the offspring's fitness, causing the useful substructure to be discarded. Conversely, an ineffective or redundant term may be retained simply because it appears in an otherwise high-performing expression. The current mechanism therefore does not reliably distinguish useful substructures from spurious ones. Since substructure reuse is a central contribution of the paper, the lack of validation of this credit-assignment strategy is a significant weakness. The ablation studies evaluate the overall effect of enabling substructure guidance, but do not establish whether the archive reliably identifies individually beneficial substructures.
  2. Insufficient Case Analysis The paper provides examples of LLM-driven mutation and substructure-guided generation, but does not sufficiently expose the actual contents and evolution of the substructure archive. In particular, it is unclear which substructures are retained over successive generations, how often they are reused, and whether they are genuinely beneficial or merely associated with high-performing complete expressions. More detailed case studies tracing the parent expression, the mutation, the extracted substructure, its archive score, and its subsequent effect on generated candidates would help substantiate the proposed mechanism.
  3. Unclear NMSE Aggregation Table 1 reports domain-level median NMSE values averaged across five runs, but the exact aggregation order across benchmark equations and independent runs is not clearly specified. Moreover, NMSE values can be highly skewed and span several orders of magnitude. Although the median is robust to outliers, it may conceal severe failures on individual equations. The paper would benefit from an explicit description of the aggregation procedure and complementary statistics, such as interquartile ranges and the proportion of problems with NMSE greater than 1, to better characterize the robustness of the reported results.
  4. Equation Numbering and Formatting. Some displayed equations in the paper are not numbered. The authors should ensure that equations presented on separate lines are numbered consistently where appropriate, following the standard formatting conventions of the conference template, to improve readability and facilitate referencing.
Questions for authors
  1. Substructure Fitness Attribution When a single LLM mutation simultaneously adds, removes, or modifies multiple mathematical terms, how does the method determine whether an individual substructure is beneficial? Is there empirical evidence that the current scoring strategy, based on the NMSE of the complete offspring expression, can reliably identify useful substructures?
  2. Clarification of the NMSE Aggregation Procedure The aggregation procedure used to obtain the domain-level NMSE values in Table 1 is not entirely clear. Is the median first computed across benchmark equations for each run and then averaged across five runs, or are the five-run results first aggregated for each equation before computing the domain-level median? Clarifying this procedure would facilitate the interpretation and reproducibility of the reported results.
  3. Full Equation Recovery Rates Across Baselines Since the ultimate goal of symbolic regression is to recover the underlying mathematical expressions, it would be valuable to report the proportion of problems for which each baseline successfully recovers the complete ground-truth equation across the evaluated datasets. This would complement the numerical error and structural similarity metrics and provide a more direct assessment of the methods' symbolic recovery capabilities.
  4. Detailed Examples of Substructure Archive Evolution The examples of substructure-guided generation are somewhat limited. A more detailed illustration of the archive throughout the evolutionary process, including the retained substructures, their associated fitness scores, reuse frequency, and effects on subsequent candidate expressions, would help clarify how the archive operates in practice and how it contributes to the overall performance of EvoMO-SR.
1 comment
AuthorsOpenReview ↗

We aknowledged the limitations of the credit assignment score for substructures; we performed additional experiments concerning the reuse of substructures; we clarified NMSE aggregation procedure and returned some additional metrics; we returned full equation recovery rates.

We thank the reviewer for recognizing the strenghts of our work, especially in the experimental efforts. Concerning the raised issues, we report in the following clarifications and additional experiments that we hope can be useful.

  1. Substructure Fitness Attribution. We agree that the credit assignment does not reflect the individual contribution of a single substructure (as already aknowledged in the conclusions). We acknowledge that the NMSE-based credit assignment mechanism may retain substructures whose individual contribution is limited or discard useful substructures occurring in poorly performing expressions. However, we would like to clarify that our goal was not to estimate the independent contributions, but rather to identify, retain and reuse candidate substructures disovered during the evolutionary process. The explicit identification and reuse of additive mathematical substructures through parent-offspring comparisons distinguishes EvoMO-SR from the main LLM-based SR approaches considered in this study. Nevertheless, the development of a score tailored to assess individual contribution is certainly a future line of work we wish to pursue.
  2. Clarification of the NMSE Aggregation Procedure: we appreciate the request for clarification. As indicated in Section 4.1 (p. 6) and the caption of Table 1, the reported domain-level values are obtained by computing the median NMSE across the benchmark problems within each domain for each independent run, and subsequently averaging these medians across the five runs. So, the order is: 1) compute the domain-level median across problems for each run, 2) average the five resulting medians. The paper complements aggregate performance with problem-level paired Wilcoxon tests (Appendix C.4, Table 7). Following your suggestion, we also report here the proportion of individual problem-run evaluations with NMSE greater than 1, comparing EvoMO-SR and LLM-SR (percentages are computed on the 5 runs for each problem)
  • Physics ID: LLM-SR 0%, ours 2.7%; OOD: LLM-SR 18.2%, ours 5.0%
  • Chemistry ID: LLM-SR 0.6%, ours 0.6%; OOD: LLM-SR 39.4%, ours 26.7%
  • Biology ID: LLM-SR 1.7%, ours 0%; OOD: LLM-SR 44.3%, ours 28.7%
  • Material Science ID: LLM-SR 0%, ours 0%; OOD: LLM-SR 7.2%, ours 0%
  • Overall: ID LLM-SR 0.5%, ours: 1.1%; OOD LLM-SR 26.7%; ours 14.4%. Therefore EvoMO-SR obtained fewer large OOD errors overall.
  1. Full Equation Recovery Rates Across Baselines: we agree that reporting the proportion of problems for which a baseline recover ground truth expressions could be beneficial. We report in the following the percentages of problems (across all 5 runs) in which a baseline obtained SA = 1. For SA_AST: uDSR 0.0%, GP-GOMEA 0.0%, PySR 0.0%, Operon 0.0%, LLM-SR 0.47%, LaSR 0.31%, EvoMO-SR 2.95%. For SA_Term: uDSR 0.62%, GP-GOMEA 0.0%, PySR 0.0%, Operon 0.0%, LLM-SR 0.16%, LaSR 0.31%, EvoMO-SR 3.57%

  2. Detailed Examples of Substructure Archive Evolution: we agree that a more detailed illustration of archive evolution would improve the interpretability of the proposed mechanism. Following this suggestion, we carried out additional experiments on oscillator2, CRK34 problem (LSR-Synth), and the P3 problem (randomly generated function), each for 5 runs. We investigated whether reused substructures were beneficial by removing the added substructure for each incorporation case. After coefficient re-fitting, the expression was evaluated computing its ID and OOD NMSE values and compared with errors of the expression with the substructure. Results shown that removing the substructure increased ID NMSE in 72.7% of cases and OOD NMSE in 52.1% of cases (higher is worse). An illustrative example is the following: for the P3 problem, in one run the substructure x22x_2^2 was extracted at generation 3 from an offspring with training NMSE = 0.39845. Although the offspring did not survive selection, the substructure remained in the archive and was subsequently reused at generation 8 to mutate another parent. The resulting offspring improved training NMSE from 0.57701 to 0.00935, survived selection, and entered the Pareto archive.

Additionally, for each substructure present in the ground truth, we recorded their improvement rate (on parent expressions) and their presence in the final expression. For oscillator2:

  • sin(t)sin(t) -> improvement 72.2%, presence 5/5.
  • v3v^3 -> improvement 43.8%, presence 5/5.
  • vxvx -> improvement 43.8%, presence 5/5
  • x⋅exp(x)x \cdot exp(x) -> improvement 100%, presence 0/5. Only appeared in 4 generations. For CRK34:
  • A2A^2 -> improvement 57.6%, presence 5/5.
  • A1/3A^{1/3} was never retained. For P3:
  • x02x_0^2 -> improvement 70%, presence 4/5
  • x1⋅sin(x1)x_1 \cdot sin(x_1) was never retained.
  • x22x_2^2 -> improvement 100%, presence 4/5.

With these further findings, we aim to demonstrate the value of introducing the substructure guidance mechanism, without, however, claiming that it is, in absolute terms, a method that consistently improves performance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.