acceptodds
Under review as a conference paper at ICLR 2027

ATLAS-SR: A Protocol-Indexed Structural Atlas for LLM-Guided Symbolic Regression

Abstract

Symbolic regression seeks interpretable mathematical expressions that capture relationships in observed data. During search, useful components can appear in candidate equations whose overall fit is poor. We present ATLAS-SR, a framework that uses large language models (LLMs) to guide the recombination of components retained from earlier candidates. The framework tests component contributions within existing equations under consistent fitting conditions. It stores components in a shared graph alongside their tested combinations and the results of these comparisons. These records help the LLM choose which components to reuse and where to try new combinations, allowing useful structures to outlive their original candidates. We evaluate ATLAS-SR on 129 LLM-SRBench tasks spanning four scientific domains with two LLM backbones. Strict out-of-distribution accuracy (\\mathrm{Acc}^{95\\%}_{0.01}) improves by 28.69–29.46 percentage points over the strongest baseline under each backbone. Symbolic accuracy, which measures recovery of the target equation structure, reaches 24.03% with either backbone, exceeding the highest baseline result of 16.28%.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
symbolic regression, large language models, scientific discovery, equation discovery, neurosymbolic AI, structure-aware search

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Rvk5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

ATLAS-SR is an LLM-guided symbolic regression framework that maintains a Structural Atlas of reusable expression components, their assembly relationships, and protocol-indexed evaluation records. Component contributions are measured through matched refitting (MR): a component is removed, the remaining expression is refitted under matched settings, and the resulting loss change is recorded. These records guide subsequent structural edits and resource allocation. Experiments on 129 LLM-SRBench tasks with two LLM backbones show substantial numerical improvements and moderate gains in symbolic recovery.

Critical strengths
  1. Meaningful research problem. Symbolic regression is an important direction for discovering interpretable mathematical relationships. Preserving useful partial discoveries is a relevant search challenge.

  2. Broad empirical evaluation. The paper includes multiple scientific domains, two LLM backbones, several baselines, extensive ablations, noise experiments, and resource accounting.

Critical weaknesses
  1. My primary concern is the limited novelty of this work. Its central idea is to retain search experience to guide subsequent search, a strategy already explored by many related methods, including those listed in lines 108–119. The paper does not establish a substantially new idea or motivation relative to these approaches. The distinctions described in lines 117–119 and 126–129 appear to be incremental implementation and procedural refinements, which I do not consider sufficient for ICLR. The contribution list in lines 85–95 further reflects this limited methodological novelty. Moreover, the paper omits the closely related SR-LLM [1], which already combines a dynamic expression library, knowledge memory, symbol evaluation, and LLM-driven updates.

  2. The core MR score in Equation (1) relies solely on changes in fitting loss after component deletion and refitting, which I consider an inadequate basis for component evaluation. Refitting does not eliminate the effects of strong coupling among coefficients and expression terms. More importantly, loss reduction can reflect noise fitting rather than meaningful structural contributions. Section 4.5 demonstrates severe degradation under noise: strict OOD accuracy drops from 85.27% to 46.51% with 1% noise, and symbolic accuracy falls to 0.78% with 5% noise. Real-world noise can be considerably more complex, and this reliance on fitting loss raises serious concerns about the method’s applicability to real data.

  3. The appendices define the Atlas state and update interfaces, but several decisions essential to reproduction remain qualitative: component extraction and retrieval, the complete initial structural library, deactivation thresholds, scheduling heuristics, and final structural eligibility and stability checks. Complete prompts and action specifications are also not provided. I could not find a code or anonymous repository link in the supplied manuscript. These choices directly affect search coverage and selection, so their omission limits both reproducibility and interpretation of the reported gains.

  4. The LLM-SR results in Table 1 of this submission differ substantially from those in Table 1 of LLM-SRBench, despite both using GPT-4o-mini. LLM-SR’s median NMSE increases from 4.12×10−64.12\times10^{-6} to 5.12×10−55.12\times10^{-5} in chemistry and from 3.06×10−63.06\times10^{-6} to 1.09×10−41.09\times10^{-4} in biology compared with LLM-SRBench—approximately 12.4× and 35.6× higher, respectively. These substantial discrepancies are not adequately explained. Budget comparability is also unclear: LLM-SRBench specifies 1,000 LLM calls per task, whereas this paper specifies 1,000 samples with four samples per prompt; these are not necessarily equivalent. Are these budgets equivalent? If not, why was the LLM-SR baseline not evaluated using the budget specified in LLM-SRBench? Without released code or a reproducibility statement, these discrepancies and budget ambiguities undermine confidence in the baseline implementations and the reported gains.

[1] SR-LLM: An Incremental Symbolic Regression Framework Driven by LLM-Based Retrieval-Augmented Generation, PNAS

Questions for authors

Please refer to weaknesses.

Reviewer_bGp6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ATLAS-SR, an LLM-guided symbolic regression framework based on structured experience memory, aiming to address the difficulty of effectively reusing valuable local structures from previously explored candidate equations. The proposed method introduces a Structural Atlas to store reusable equation components, their assembly relations, and corresponding evaluation evidence. It further employs Matched Refit to assess component contributions and Protocol Indexing to ensure consistency across evaluation settings, thereby guiding subsequent structural composition and equation search.

Critical strengths
  1. The research problem is meaningful. Effectively leveraging useful local structures from historical candidate equations, rather than relying solely on the fitting scores of complete equations, is an important problem worth investigating in LLM-based symbolic regression.
Critical weaknesses
  1. The methodological novelty of Structural Atlas remains unclear. While the framework organizes component relations through a typed hypergraph and associates evaluation results with specific protocols, its underlying ideas overlap with existing term-level refinement and memory methods. The authors should clarify whether these design choices introduce fundamentally new search capabilities or primarily improve the organization and reuse of existing search experience.
  2. A central motivation of ATLAS-SR is to improve the recovery of complex equation structures through the preservation and recombination of local components. However, the RCB-stratified results in Appendix Table 9 show that its gains are mainly concentrated on low-complexity tasks (RCB = 0–1). For the 94 higher-complexity tasks (RCB = 2–4), ATLAS-SR recovers only 7 target equations with DeepSeek-V4-Flash, compared to 9 recovered by IGSR. These results raise concerns about the method's effectiveness on complex equations. Further analysis of failure cases and additional experiments on high-complexity tasks would help substantiate the claimed advantages in structural recovery.
Questions for authors
  1. According to Appendix Table 9, ATLAS-SR does not outperform IGSR on tasks with RCB ≥ 2. Given that a central motivation of this work is to improve complex structural composition through component reuse, how do the authors explain this result?
  2. Regarding the reliability of Matched Refit: When multiple components are highly correlated, or when incorrect components can closely approximate the true operators, MR scores may not accurately reflect their true structural value. Have the authors evaluated how component correlations affect MR scores and subsequent search decisions? Furthermore, what motivates the choice of the fixed thresholds d > 10 and d < 1.2, and how sensitive is the final performance to these thresholds?
Reviewer_6Qq5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ATLAS-SR: an LLM-based symbolic regression method. It stores components from past candidates in a shared graph, together with deletion-and-refit evidence of how much each component contributes under a recorded fitting protocol. An LLM uses this record each round to prioritize candidates, edits and tests. On LSR-Synth benchmark, the method shows large gains in OOD and symbolic accuracy over existing baselines.

Critical strengths
  • Results look strong and consistent across domains and backbones, using very few LLM calls.

  • The ablations are fairly thorough and test the main design choices. The failure analysis (incomplete assembly, operator shadows) is informative and helps explain why exact recovery remains difficult.

Critical weaknesses
  • The methodological novelty seems limited. Term-level credit through deletion and refitting, reuse of terms across candidates, and iterative term-level refinement have been explored in prior work (e.g., IGSR, PTB-Search, DecAEvolve, RESTART). The main addition appears to be storing this evidence across rounds and extending it to non-additive structures.

  • Some evaluation results are hard to interpret. The variant without any LLM outperforms PySR by a large margin, and the baseline configurations and budgets are not documented. This raises questions about how strong the baselines are and where the gains come from.

  • The contribution of the LLM and of domain knowledge is not clearly separated from the search and fitting machinery, and comparisons are not compute-matched.

  • Several details needed for reproducibility are missing, and the evaluation leaves out complexity, run-to-run variance and non-synthetic data.

Questions for authors
  1. Could the authors add a representative prompt and JSON plan to the paper, and report how many evaluated or selected candidates come from LLM-prioritized actions versus default editor exploration?

  2. Have the authors run a version with variable names and task descriptions removed? This would show how much the gains depend on domain knowledge.

  3. The no-LLM variant reaches 34.1% strict OOD accuracy, versus 16.3% for PySR. Could the authors explain this gap, and list the PySR and PhySO configurations used (iterations, populations, operators, time limits)? Could you also report PySR, IGSR and the no-LLM variant under the same CPU or wall-time budget as the full method?

  4. How are the 32 seed structures built, and are they task-specific? What do the structural checks in final selection test, and how often do they change the output?

  5. Could the authors report results over multiple runs, and compare the complexity of final expressions across methods?

  6. Could the authors clarify what is conceptually new relative to IGSR’s refit-aware deletion scores, PTB-Search’s cross-candidate term dictionary, DecAEvolve and RESTART? A direct comparison with those would help.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.