- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes an agentic framework for symbolic regression based on two components. 1. Cotras: a trajectory based search control by comparing structural transitions and ma-hase that checks which structures are supported by the data. They evaluate it on 5 real world SR problems against existing methods.
Critical strengths
- The motivation that search trajectories should be leveraged is a step in correct direction.
Critical weaknesses
- While the motivation is correct, the paper does not establish the reason for the same with prior analysis. My main concern is authors state in the introduction that "Existing agents retain limited structured information beyond candidate expressions", "Unsuccessful candidates are often discarded without extracting reusable knowledge" and "Scientific knowledge is frequently used for post hoc validation". I agree that by design these might be limitations of existing methods. However, it is not established by any experiments or analysis.
- The proposed components feel like more structured forms of feedback and memory rather than solutions to some limitation.
- The evaluation is on 5 real world SR problems with non standard metrics and more importantly lack the established baselines such as LLMSRBench, SRBench, etc. While all benchmarks are not expected, evaluation seems unconvincing when its only on 5 tasks proposed by the paper itself.
Questions for authors
see weaknesses
5 comments
Response to Reviewer Ua3k (Part 2)
W3: Evaluation on five space-physics tasks is insufficient without established symbolic regression benchmarks.
A3: Thank you for this valuable suggestion. We have added experiments on LSR-Synth from LLM-SRBench covering 24 Biology and 25 Materials problems. We compare both non-LLM baselines and LLM-based methods including SR-Scientist, LLM-SR, and LaSR. These experiments evaluate both in-distribution (ID) and out-of-distribution (OOD) performance without activating domain-specific physical or mechanistic constraints.
| Method | All ACC₀.₀₁ | All ACC₀.₀₀₁ | Mat. ACC₀.₀₁ | Mat. ACC₀.₀₀₁ | Bio. ACC₀.₀₁ | Bio. ACC₀.₀₀₁ |
|---|---|---|---|---|---|---|
| Without LLMs | ||||||
| GPLearn | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| E2E | 0.68 | 0.00 | 1.33 | 0.00 | 0.00 | 0.00 |
| uDSR | 28.57 | 14.28 | 28.00 | 20.00 | 29.17 | 8.33 |
| PySR | 39.45 | 14.29 | 53.33 | 22.67 | 25.00 | 5.56 |
| Qwen3.5-ID | ||||||
| LLM-SR | 8.16 | 0.00 | 12.00 | 0.00 | 4.17 | 0.00 |
| LaSR | 38.78 | 16.33 | 64.00 | 24.00 | 12.50 | 8.33 |
| SR-Scientist | 59.18 | 34.69 | 72.00 | 52.00 | 45.83 | 16.67 |
| EvoTrace | 79.59 | 55.10 | 72.00 | 60.00 | 87.50 | 50.00 |
| Qwen3.5-OOD | ||||||
| LLM-SR | 30.61 | 12.24 | 52.00 | 20.00 | 8.33 | 4.17 |
| LaSR | 48.98 | 28.57 | 84.00 | 48.00 | 12.50 | 8.33 |
| SR-Scientist | 55.10 | 40.82 | 76.00 | 64.00 | 33.33 | 16.67 |
| EvoTrace | 61.22 | 42.86 | 76.00 | 64.00 | 45.83 | 20.83 |
| Qwen3.6-ID | ||||||
| LLM-SR | 16.33 | 4.08 | 28.00 | 8.00 | 4.17 | 0.00 |
| SR-Scientist | 63.27 | 44.90 | 76.00 | 60.00 | 50.00 | 29.17 |
| EvoTrace | 71.43 | 55.10 | 80.00 | 68.00 | 62.50 | 41.67 |
| Qwen3.6-OOD | ||||||
| LLM-SR | 26.53 | 16.33 | 48.00 | 28.00 | 4.17 | 4.17 |
| SR-Scientist | 61.22 | 46.94 | 76.00 | 68.00 | 45.83 | 25.00 |
| EvoTrace | 65.31 | 53.06 | 76.00 | 68.00 | 54.17 | 37.50 |
Notes. ACC values are percentages. Subscripts 0.01 and 0.001 denote 1% and 0.1% relative-error tolerances; success requires more than 95% of test points within tolerance. All = Overall (49 problems); Mat. = Materials (25); Bio. = Biology (24). Bold marks the best accuracy within each backbone/split block, including ties.
EvoTrace achieves the highest Overall accuracy in all eight backbone/split/tolerance comparisons. Under Qwen3.5, its ID accuracy reaches 79.59%/55.10%, exceeding SR-Scientist by 20.41 percentage points at both tolerances. Under Qwen3.6, EvoTrace achieves 71.43%/55.10% on ID and 65.31%/53.06% on OOD, outperforming SR-Scientist across all four aggregate metrics.
The domain-level results further demonstrate strong equation recovery. On Biology under Qwen3.5, EvoTrace reaches 87.50% ID accuracy at the 1% tolerance and 50.00% at the stricter 0.1% tolerance. On Materials under Qwen3.6, it achieves 80.00%/68.00% ID accuracy. Together with the OOD improvements, these results demonstrate effective recovery across two additional scientific domains and support generalization beyond the original space-physics applications. The results and experimental details are included in Appendix D.7, Table 19, and Appendix C.
rebuttal review
Thank you for the reply. W1: While i appreciate that narrowing the claims will resolve the immediate review. I dont see the contribution of paper beyond a more structured memory. As mentioned in the review, the direction that prior agents dont leverage the feedback to the fullest is plausible. However, the rebuttal does not address it as the experiments are not with prior agents but with an LLM. The agents already have a strctuted memory so its beneficial to the community if the paper can show this causally. The current rebuttal shows that for vanilla LLM and even in that the qwen 3.5/3.6 models that are known to be weak to begin with. My original concern was whether existing symbolic regression agents fail to make use of feedback they already collect. My concern is still not addressed here and the rebuttal provides no additional information along that direction
W2: I agree that the components help. However, i still am unclear why do they help. The ablation shows that adding MA-HASE helps on that task. However, a simple filter using the same scientific constraints could get similar gains. Similarily, CoTraS stores edit outcome records and turns them into search feedback. The ablation removes both, so it cannot tell whether the gain comes from the records themselves or from how CoTraS uses them. To sum up, i understand and agree that these components help, however i dont understand why do they help and what do they provide that the prior methods didnt. Neither the paper nor the rebuttal addresses this.
W3: Thank you for adding the new experiments. I recommend adding more splits of LLMSRBench or other established benchmarks. I am not asking for more tasks. A subset of existing benchmarks is sufficient. That being said, I am highly sceptical of these results. LLMSR, LaSR scores lower ID than OOD. LLMSR also scores near zero while the published numbers are differ significantly with weaker backbones. Can the authors confirm that the methods were run correctly? Apart from this W3 is no longer a concern and i appreciate the effort.
Response to Reviewer Ua3k (Part 1)
W1: The limitations motivating EvoTrace are not sufficiently supported by prior analysis.
A1: Thank you for pointing out this gap. We will revise the Introduction to avoid overly broad statements and distinguish observations about existing designs from the hypotheses motivating our method.
To provide empirical evidence, we added memory comparisons on 49 LSR-Synth problems: 24 Biology and 25 Materials problems. We compare ordinary search history (H), paired modification memory (H+P), and independent failed-family memory (H+F), without activating domain-specific mechanistic constraints. P preserves the reference expression, its modification, and the evaluated outcome; F separately organizes unsuccessful formula families.
Notes. Values are Overall problem-level success rates (%). Subscripts 0.01 and 0.001 denote 1% and 0.1% relative-error tolerances. Bold indicates the best result within each backbone, including ties.
Paired modification memory improves all eight aggregate scores over ordinary history. At the 1% tolerance, ID gains are 10.20 and 6.12 percentage points under Qwen3.5 and Qwen3.6, respectively; OOD gains are 8.16 points under both backbones.
Independent failed-family memory also provides useful guidance: H+F improves both OOD scores under both backbones and improves strict ID accuracy under Qwen3.5. Its ID effects are less consistent, showing that different memory representations are not interchangeable. H+P matches or exceeds H+F in all eight comparisons, supporting the value of preserving evaluated modifications rather than only archiving failures.
These results provide evidence for the memory-design motivation, without implying that every existing agent discards unsuccessful experience. The full domain-level results and definitions are included in Appendix D.5, Tables 16–17, and Appendix C.2.
W2: The proposed components appear to be structured feedback and memory rather than solutions to an established limitation.
A2: Thank you for this comment. CoTraS and MA-HASE are complementary contributions that form a closed-loop system for learning from evaluated search experience and controlling subsequent hypothesis evolution.
When search relies mainly on candidate-level scores, an improvement does not explicitly reveal which structural modification produced it or under what conditions that modification remains useful. This can lead to repeated unsuccessful exploration or the continuation of edits that improve local fit but damage extrapolation or coupled relationships. CoTraS addresses this problem by retaining reference–edit–outcome evidence and compiling it into guidance about which structural directions to pursue or avoid. MA-HASE complements this guidance by checking scientific admissibility during candidate generation and retention. Together, they connect empirical modification evidence to both future exploration and candidate acceptance.
The physical-task ablations demonstrate the practical benefit of this coupling. With MA-HASE retained, adding CoTraS reduces sunspot MSE from 3998.17 to 1422.16 (64.4%) and Fe X ratio MSE from 0.14650 to 0.012333 (91.6%). The latter improvement occurs despite the ablated version achieving lower individual line-wise errors, illustrating why isolated fit scores are insufficient for coupled recovery. Conversely, with CoTraS retained, adding MA-HASE reduces differential-rotation MSE from 4.71×10⁻⁴ to 2.26×10⁻⁵ (95.2%). These results, reported in Appendix D.4, Table 15, support the two modules as complementary search mechanisms rather than additional memory storage alone.