- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
ATLAS-SR is an LLM-guided symbolic regression framework that maintains a Structural Atlas of reusable expression components, their assembly relationships, and protocol-indexed evaluation records. Component contributions are measured through matched refitting (MR): a component is removed, the remaining expression is refitted under matched settings, and the resulting loss change is recorded. These records guide subsequent structural edits and resource allocation. Experiments on 129 LLM-SRBench tasks with two LLM backbones show substantial numerical improvements and moderate gains in symbolic recovery.
Critical strengths
-
Meaningful research problem. Symbolic regression is an important direction for discovering interpretable mathematical relationships. Preserving useful partial discoveries is a relevant search challenge.
-
Broad empirical evaluation. The paper includes multiple scientific domains, two LLM backbones, several baselines, extensive ablations, noise experiments, and resource accounting.
Critical weaknesses
-
My primary concern is the limited novelty of this work. Its central idea is to retain search experience to guide subsequent search, a strategy already explored by many related methods, including those listed in lines 108–119. The paper does not establish a substantially new idea or motivation relative to these approaches. The distinctions described in lines 117–119 and 126–129 appear to be incremental implementation and procedural refinements, which I do not consider sufficient for ICLR. The contribution list in lines 85–95 further reflects this limited methodological novelty. Moreover, the paper omits the closely related SR-LLM [1], which already combines a dynamic expression library, knowledge memory, symbol evaluation, and LLM-driven updates.
-
The core MR score in Equation (1) relies solely on changes in fitting loss after component deletion and refitting, which I consider an inadequate basis for component evaluation. Refitting does not eliminate the effects of strong coupling among coefficients and expression terms. More importantly, loss reduction can reflect noise fitting rather than meaningful structural contributions. Section 4.5 demonstrates severe degradation under noise: strict OOD accuracy drops from 85.27% to 46.51% with 1% noise, and symbolic accuracy falls to 0.78% with 5% noise. Real-world noise can be considerably more complex, and this reliance on fitting loss raises serious concerns about the method’s applicability to real data.
-
The appendices define the Atlas state and update interfaces, but several decisions essential to reproduction remain qualitative: component extraction and retrieval, the complete initial structural library, deactivation thresholds, scheduling heuristics, and final structural eligibility and stability checks. Complete prompts and action specifications are also not provided. I could not find a code or anonymous repository link in the supplied manuscript. These choices directly affect search coverage and selection, so their omission limits both reproducibility and interpretation of the reported gains.
-
The LLM-SR results in Table 1 of this submission differ substantially from those in Table 1 of LLM-SRBench, despite both using GPT-4o-mini. LLM-SR’s median NMSE increases from to in chemistry and from to in biology compared with LLM-SRBench—approximately 12.4× and 35.6× higher, respectively. These substantial discrepancies are not adequately explained. Budget comparability is also unclear: LLM-SRBench specifies 1,000 LLM calls per task, whereas this paper specifies 1,000 samples with four samples per prompt; these are not necessarily equivalent. Are these budgets equivalent? If not, why was the LLM-SR baseline not evaluated using the budget specified in LLM-SRBench? Without released code or a reproducibility statement, these discrepancies and budget ambiguities undermine confidence in the baseline implementations and the reported gains.
[1] SR-LLM: An Incremental Symbolic Regression Framework Driven by LLM-Based Retrieval-Augmented Generation, PNAS
Questions for authors
Please refer to weaknesses.