acceptodds
Withdrawn from ICLR 2027

Equation Discovery Through Collectively Evolving Effective Hypothesis Generators

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Deriving extrapolatable symbolic laws from empirical observations remains a central bottleneck in AI-driven scientific discovery. We propose collective reasoning intelligence for symbolic optimization (CRISO), a multi-agent framework for autonomous equation discovery without expert feedback, task-specific finetuning, or external scientific tools and knowledge bases. CRISO distills collectively selected best hypothesis into reusable scientific statements, accumulates them as shared context, and broadcasts to a population of reasoning agents, thereby implementing a collectively evolving symbolic regression. Consequently, CRISO evolves the hypothesis search toward promising regions without additional finetuning of the backbone LLM. Across ten nonlinear, stochastic, or previously uncharacterized scientific systems, CRISO outperformed or matched state-of-the-art combinatorial and recent LLM-based symbolic regression methods, including under out-of-distribution evaluation. On an in-house chemical reactor whose dynamics lie beyond the backbone LLM's prior knowledge, CRISO derived a symbolic equation with 38.02% lower generalization error than all baselines, demonstrating its scientific discovery capability beyond static LLM priors.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Symbolic Regression, AI4Science, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_XWrjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CRISO, a multi-agent framework for symbolic regression. CRISO combines LLM-based hypothesis generation, AST-based hypothesis evaluation, and a global knowledge distillation mechanism that extracts reusable scientific statements from the best-performing equation and shares them across agents. Experiments on ten scientific regression tasks demonstrate strong predictive performance, including improved out-of-distribution generalization on six benchmarks.

Critical strengths

1, Strong empirical performance. CRISO achieves strong results across ten symbolic regression tasks spanning physics, chemistry, and biology, outperforming several competitive baselines, including LLM-SR, FunctionEvolve, and STRIDE. The paper additionally evaluates out-of-distribution generalization on six tasks with known ground truth equations.

2, No task-specific LLM fine-tuning required. CRISO operates with frozen backbone LLMs. The authors also provide ablation studies and experiments with different backbone LLMs to support the effectiveness and generalizability of the framework.

3, A useful real-world benchmark. The NOMC dataset provides a valuable real-world evaluation setting for LLM-based symbolic regression, reducing the risk of data leakage during pretraining.

Critical weaknesses

1, Insufficient differentiation from existing methods. First, It is unclear how CRISO's multi-agent framework fundamentally differs from the multi-island and multi-population approaches employed by LLM-SR[1] and LaSR[2], respectively. Both prior methods already incorporate multiple independently evolving search populations, along with mechanisms for information exchange. Second, the accumulation and reuse of knowledge have already been explored in recent works, such as LaSR[2]'s concept library, RESTART[3]'s structure library, and DrSR[4]'s idea library. The paper should more explicitly discuss how CRISO differs from these approaches to better establish its methodological novelty.

2, Insufficient justification for excluding LLM-SRBench[5]. Although the authors briefly discuss the differences between their evaluation datasets and LLM-SRBench, which also includes a physics-related subset. LLM-SRBench is specifically designed to evaluate symbolic regression beyond LLM memorization with total 239 tasks. Additional experiments on LLM-SRBench, or a more detailed justification for its exclusion, would strengthen the empirical evaluation.

3, Lack of symbolic accuracy evaluation. The paper reports WMAPE and L2 relative error and provides comparisons of selected discovered equations. However, these evaluations mainly measure predictive accuracy and equation complexity rather than whether the underlying symbolic structures are correctly recovered. Since six tasks have known ground-truth equations, the authors could additionally report symbolic accuracy or other commonly used evaluation metrics, such as Acc0.1Acc_{0.1} in LLM-SRBench, to provide a more comprehensive evaluation of equation discovery performance.

Minor:

1, The ablation studies do not isolate the contribution of cumulative knowledge accumulation.

2, Authors report N/A results due to unresolved runtime errors, but the paper does not provide sufficient details about these failures or the attempts made to resolve them. This lack of clarification raises concerns about the fairness of the baseline comparisons.

[1] Shojaee, P., Meidani, K., Gupta, S., Barati Farimani, A., and Reddy, C. K. (2025). LLM-SR: Scientific Equation Discovery via Programming with Large Language Models. ICLR 2025.

[2] Grayeli, A., Sehgal, A., Costilla-Reyes, O., Cranmer, M., and Chaudhuri, S. (2024). Symbolic Regression with a Learned Concept Library. NeurIPS 2024.

[3] Li, Y., and Pan, S. J. (2026). Robust Equation Structure Learning with Adaptive Refinement. ICLR 2026.

[4] Wang, R., Wang, B., Li, K., Zhang, Y., and Cheng, J. (2025). DrSR: LLM Based Scientific Equation Discovery with Dual Reasoning from Data and Experience. arXiv preprint arXiv:2506.04282.

[5] Shojaee, P., Nguyen, N.-H., Meidani, K., Barati Farimani, A., Doan, K. D., and Reddy, C. K. (2025). LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models. ICML 2025.

Questions for authors

See Weaknesses.

Reviewer_QBhtOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CRISO, a multi-agent LLM framework for equation discovery, where each agent writes equation programs, and a separate LLM distills the best equation into natural-language knowledge that is shared with all agents. The authors report lower WMAPE than several baselines on ten problems, including an in-house chemical reactor.

Critical strengths
  1. The method works well on the in-house NOMC reactor, an anti-memorization problem with lower WMAPE and a few extra costs in time.
  2. The CRISO_ind ablation shows that sharing the distilled knowledge gives a clear gain over independent agents with the same budget.
Critical weaknesses
  1. The innovation is marginal. Distilling good equations into natural-language knowledge is close to LaSR and DrSR, and DrSR is neither cited nor compared. Adding AST depth and parameter count as complexity regularization is common in symbolic regression and already used by FunctionEvolve and STRIDE.

  2. The evaluation uses only error metrics (WMAPE, L2RE), not symbolic accuracy, so it cannot reflect the actual explicability of the discovered equations. For example, the ECBG and Chi2PDF equations have low error but a different structure from the true equations, and AST depth does not capture this.

  3. Each agent proposes hypotheses only by LLM sampling, without a search algorithm. The cost therefore grows with K × L (5,000–10,000 calls, about 7.7 hours per run), and many proposals may be duplicates since all agents receive the same knowledge. The paper does not report token counts, accuracy-versus-compute curves, or the wall-clock time of PySR and GenSR.

  4. The paper discloses that all LLM-based methods use Mixtral 8x7B, but not the quantization, decoding settings, or GPT-5.6 Luna details needed for a fair comparison. Some baselines also seem not to work properly: FunctionEvolve spends about 180 of its 230 calls on retries, STRIDE has runtime errors on NDO, and PySR’s results look nearly constant.

Questions for authors
  1. How are the equation coefficients fitted in CRISO, and how were K, L, and R chosen for each problem? R is not reported anywhere.

  2. Can the authors report symbolic accuracy (e.g., recovery rate) for the six problems with known equations, and compare with DrSR on LLM-SRBench?

  3. What quantization, temperature, and token limits were used for each LLM-based method, and how was the budget defined for GPT-5.6 Luna?

Reviewer_xpYNOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes CRISO, a multi-agent equation discovery framework that summarizes the best equation into natural-language guidance and shares it across agents. Results across ten problems are good, but I’m not yet convinced the contribution goes substantially beyond existing concept-guided LLM-based search

Critical strengths
  • Useful extra analyses on OOD tests, AST-depth comparison with the true equations, etc.
  • Useful ablation study comparing independent agents, direct equation sharing, and distilled guidance.
Critical weaknesses
  • Ten tasks are a limited basis for general claims. Could the authors include results on a larger benchmark LLM-SRBench [1] which is standard in literature for LLM-based equation discovery and report equation-recovery rates or symbolic accuracy alongside prediction errors / numeric accuracy metrics?

  • Distilling good equations into text and feeding them back is close to shared-exemplar populations in LLM-SR and other equation disocvery methods.

  • In my view some of the claims feel stronger than the evidence provided for them. “Evolving hypothesis generators” here means updating prompts and memory. What additional insight does this framing provide beyond ordinary inference-time search with better feedback?

  • Appendix B reports frequent invalid outputs for FuncEv, and some baseline runs fail entirely. How much of the gain comes from better search versus better compatibility with the chosen backbone?

  • Table 8 implies 5,000 candidate generations for most tasks, while Table 9 reports 10,200 total calls for CRISO. Can the authors clarify per-task tokens, evaluations, and runtime?

  • Replacing AST depth with code length changes the complexity objective too. Can the authors separate the benefit of the representation from the choice and scaling of the penalty?

[1] LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models, ICML, 2025

Questions for authors

check above section and some questions below:

  • Was the LLM-call budget matched across methods for fair compaction? Current writeup is a bit confusing. Appendix E says 10k calls but Table 8 says mostly 5k.

  • How do you separate discovery from memorization on textbook or public equations? thats' why I think evaluation on the LLM-SRBench as a benchmark designed to avoid memorization is needed.

  • Also I wonder in the proposed method, does distilling only the single best equation collapse diversity? How does it compare with top-k distillation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.