acceptodds
Under review as a conference paper at ICLR 2027

EvoTrace: Learning to Evolve Scientific Hypotheses from Trajectories of Agentic Symbolic Regression

Abstract

Discovering concise and interpretable mathematical laws from observational data has conventionally been addressed through symbolic regression (SR). Recent advances in large language models (LLMs) have enabled this task to be formulated as an agentic search process that iteratively proposes, evaluates, and refines candidate formulas, often achieving superior performance. However, existing LLM-based approaches often fail to learn effectively from accumulated search experience: unsuccessful hypotheses are seldom reused, scalar feedback provides little guidance for structural revision, and exploration repeatedly revisits unproductive regions of the search space. This paper proposes EvoTrace, an agentic SR framework that distills actionable knowledge from search trajectories to drive mechanism-guided hypothesis evolution. By contrasting successful and unsuccessful trajectories, EvoTrace identifies structural modifications associated with performance improvements and uses them to guide subsequent hypothesis generation. It further integrates mechanism-anchored constraints with evaluation feedback to determine whether candidate hypotheses should be retained, revised, or rejected. Experiments across five key space-physics applications, including sunspot activity, plasma pressure, and lunar-tide signals, demonstrate that EvoTrace achieves stronger overall performance than existing LLM-based and non-LLM symbolic regression methods across evaluations of predictive accuracy, formula simplicity, extrapolation capability, and physical interpretability.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
symbolic regression, scientific equation discovery, large language models, agentic AI, trajectory-guided search, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Ua3kOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an agentic framework for symbolic regression based on two components. 1. Cotras: a trajectory based search control by comparing structural transitions and ma-hase that checks which structures are supported by the data. They evaluate it on 5 real world SR problems against existing methods.

Critical strengths
  1. The motivation that search trajectories should be leveraged is a step in correct direction.
Critical weaknesses
  1. While the motivation is correct, the paper does not establish the reason for the same with prior analysis. My main concern is authors state in the introduction that "Existing agents retain limited structured information beyond candidate expressions", "Unsuccessful candidates are often discarded without extracting reusable knowledge" and "Scientific knowledge is frequently used for post hoc validation". I agree that by design these might be limitations of existing methods. However, it is not established by any experiments or analysis.
  2. The proposed components feel like more structured forms of feedback and memory rather than solutions to some limitation.
  3. The evaluation is on 5 real world SR problems with non standard metrics and more importantly lack the established baselines such as LLMSRBench, SRBench, etc. While all benchmarks are not expected, evaluation seems unconvincing when its only on 5 tasks proposed by the paper itself.
Questions for authors

see weaknesses

5 comments
AuthorsOpenReview ↗

Response to Reviewer Ua3k (Part 1)

W1: The limitations motivating EvoTrace are not sufficiently supported by prior analysis.

A1: Thank you for pointing out this gap. We will revise the Introduction to avoid overly broad statements and distinguish observations about existing designs from the hypotheses motivating our method.

To provide empirical evidence, we added memory comparisons on 49 LSR-Synth problems: 24 Biology and 25 Materials problems. We compare ordinary search history (H), paired modification memory (H+P), and independent failed-family memory (H+F), without activating domain-specific mechanistic constraints. P preserves the reference expression, its modification, and the evaluated outcome; F separately organizes unsuccessful formula families.

BackboneMemoryID ACC₀.₀₁ID ACC₀.₀₀₁OOD ACC₀.₀₁OOD ACC₀.₀₀₁
Qwen3.5H53.0622.4555.1034.69
Qwen3.5H+F51.0230.6159.1840.82
Qwen3.5H+P63.2730.6163.2740.82
Qwen3.6H57.1442.8655.1038.78
Qwen3.6H+F57.1434.6959.1840.82
Qwen3.6H+P63.2748.9863.2744.90

Notes. Values are Overall problem-level success rates (%). Subscripts 0.01 and 0.001 denote 1% and 0.1% relative-error tolerances. Bold indicates the best result within each backbone, including ties.

Paired modification memory improves all eight aggregate scores over ordinary history. At the 1% tolerance, ID gains are 10.20 and 6.12 percentage points under Qwen3.5 and Qwen3.6, respectively; OOD gains are 8.16 points under both backbones.

Independent failed-family memory also provides useful guidance: H+F improves both OOD scores under both backbones and improves strict ID accuracy under Qwen3.5. Its ID effects are less consistent, showing that different memory representations are not interchangeable. H+P matches or exceeds H+F in all eight comparisons, supporting the value of preserving evaluated modifications rather than only archiving failures.

These results provide evidence for the memory-design motivation, without implying that every existing agent discards unsuccessful experience. The full domain-level results and definitions are included in Appendix D.5, Tables 16–17, and Appendix C.2.

W2: The proposed components appear to be structured feedback and memory rather than solutions to an established limitation.

A2: Thank you for this comment. CoTraS and MA-HASE are complementary contributions that form a closed-loop system for learning from evaluated search experience and controlling subsequent hypothesis evolution.

When search relies mainly on candidate-level scores, an improvement does not explicitly reveal which structural modification produced it or under what conditions that modification remains useful. This can lead to repeated unsuccessful exploration or the continuation of edits that improve local fit but damage extrapolation or coupled relationships. CoTraS addresses this problem by retaining reference–edit–outcome evidence and compiling it into guidance about which structural directions to pursue or avoid. MA-HASE complements this guidance by checking scientific admissibility during candidate generation and retention. Together, they connect empirical modification evidence to both future exploration and candidate acceptance.

The physical-task ablations demonstrate the practical benefit of this coupling. With MA-HASE retained, adding CoTraS reduces sunspot MSE from 3998.17 to 1422.16 (64.4%) and Fe X ratio MSE from 0.14650 to 0.012333 (91.6%). The latter improvement occurs despite the ablated version achieving lower individual line-wise errors, illustrating why isolated fit scores are insufficient for coupled recovery. Conversely, with CoTraS retained, adding MA-HASE reduces differential-rotation MSE from 4.71×10⁻⁴ to 2.26×10⁻⁵ (95.2%). These results, reported in Appendix D.4, Table 15, support the two modules as complementary search mechanisms rather than additional memory storage alone.

AuthorsOpenReview ↗

Response to Reviewer Ua3k (Part 2)

W3: Evaluation on five space-physics tasks is insufficient without established symbolic regression benchmarks.

A3: Thank you for this valuable suggestion. We have added experiments on LSR-Synth from LLM-SRBench covering 24 Biology and 25 Materials problems. We compare both non-LLM baselines and LLM-based methods including SR-Scientist, LLM-SR, and LaSR. These experiments evaluate both in-distribution (ID) and out-of-distribution (OOD) performance without activating domain-specific physical or mechanistic constraints.

MethodAll ACC₀.₀₁All ACC₀.₀₀₁Mat. ACC₀.₀₁Mat. ACC₀.₀₀₁Bio. ACC₀.₀₁Bio. ACC₀.₀₀₁
Without LLMs
GPLearn0.000.000.000.000.000.00
E2E0.680.001.330.000.000.00
uDSR28.5714.2828.0020.0029.178.33
PySR39.4514.2953.3322.6725.005.56
Qwen3.5-ID
LLM-SR8.160.0012.000.004.170.00
LaSR38.7816.3364.0024.0012.508.33
SR-Scientist59.1834.6972.0052.0045.8316.67
EvoTrace79.5955.1072.0060.0087.5050.00
Qwen3.5-OOD
LLM-SR30.6112.2452.0020.008.334.17
LaSR48.9828.5784.0048.0012.508.33
SR-Scientist55.1040.8276.0064.0033.3316.67
EvoTrace61.2242.8676.0064.0045.8320.83
Qwen3.6-ID
LLM-SR16.334.0828.008.004.170.00
SR-Scientist63.2744.9076.0060.0050.0029.17
EvoTrace71.4355.1080.0068.0062.5041.67
Qwen3.6-OOD
LLM-SR26.5316.3348.0028.004.174.17
SR-Scientist61.2246.9476.0068.0045.8325.00
EvoTrace65.3153.0676.0068.0054.1737.50

Notes. ACC values are percentages. Subscripts 0.01 and 0.001 denote 1% and 0.1% relative-error tolerances; success requires more than 95% of test points within tolerance. All = Overall (49 problems); Mat. = Materials (25); Bio. = Biology (24). Bold marks the best accuracy within each backbone/split block, including ties.

EvoTrace achieves the highest Overall accuracy in all eight backbone/split/tolerance comparisons. Under Qwen3.5, its ID accuracy reaches 79.59%/55.10%, exceeding SR-Scientist by 20.41 percentage points at both tolerances. Under Qwen3.6, EvoTrace achieves 71.43%/55.10% on ID and 65.31%/53.06% on OOD, outperforming SR-Scientist across all four aggregate metrics.

The domain-level results further demonstrate strong equation recovery. On Biology under Qwen3.5, EvoTrace reaches 87.50% ID accuracy at the 1% tolerance and 50.00% at the stricter 0.1% tolerance. On Materials under Qwen3.6, it achieves 80.00%/68.00% ID accuracy. Together with the OOD improvements, these results demonstrate effective recovery across two additional scientific domains and support generalization beyond the original space-physics applications. The results and experimental details are included in Appendix D.7, Table 19, and Appendix C.

Reviewer_Ua3kOpenReview ↗

rebuttal review

Thank you for the reply. W1: While i appreciate that narrowing the claims will resolve the immediate review. I dont see the contribution of paper beyond a more structured memory. As mentioned in the review, the direction that prior agents dont leverage the feedback to the fullest is plausible. However, the rebuttal does not address it as the experiments are not with prior agents but with an LLM. The agents already have a strctuted memory so its beneficial to the community if the paper can show this causally. The current rebuttal shows that for vanilla LLM and even in that the qwen 3.5/3.6 models that are known to be weak to begin with. My original concern was whether existing symbolic regression agents fail to make use of feedback they already collect. My concern is still not addressed here and the rebuttal provides no additional information along that direction

W2: I agree that the components help. However, i still am unclear why do they help. The ablation shows that adding MA-HASE helps on that task. However, a simple filter using the same scientific constraints could get similar gains. Similarily, CoTraS stores edit outcome records and turns them into search feedback. The ablation removes both, so it cannot tell whether the gain comes from the records themselves or from how CoTraS uses them. To sum up, i understand and agree that these components help, however i dont understand why do they help and what do they provide that the prior methods didnt. Neither the paper nor the rebuttal addresses this.

W3: Thank you for adding the new experiments. I recommend adding more splits of LLMSRBench or other established benchmarks. I am not asking for more tasks. A subset of existing benchmarks is sufficient. That being said, I am highly sceptical of these results. LLMSR, LaSR scores lower ID than OOD. LLMSR also scores near zero while the published numbers are differ significantly with weaker backbones. Can the authors confirm that the methods were run correctly? Apart from this W3 is no longer a concern and i appreciate the effort.

Reviewer_4YxPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes EvoTrace, an agentic symbolic regression framework that learns from search trajectories. Its core idea is to identify effective structural editing strategies by contrasting successful and unsuccessful equation modification trajectories, while incorporating physical constraints to dynamically adapt the hypothesis search space. The framework consists of CoTraS, MA-HASE, and several auxiliary components for managing search evidence.

Critical strengths

Clear research motivation. Unlike conventional approaches that primarily retain candidate equations and their fitness scores, this work additionally records structural modifications, evaluation outcomes, and failure patterns, enabling the search process to exploit richer historical information.

Meaningful methodological contribution. By combining contrastive trajectory analysis with physics-based constraints, the proposed framework provides a structured search strategy for the iterative evolution of scientific hypotheses.

**Relatively comprehensive experiments. **The method is evaluated on five real-world space-physics tasks using three LLM backbones, with three independent random seeds. The evaluation considers predictive accuracy, extrapolation capability, physical consistency, and expression complexity. The ablation studies also provide evidence supporting the effectiveness of the main components.

Critical weaknesses

Insufficient isolation of the core contributions. CoTraS, MA-HASE, and the auxiliary evidence-management mechanisms are closely coupled. Although the paper includes component-wise ablations, it remains difficult to determine whether the observed performance improvements primarily arise from trajectory learning or from the additional physical priors and candidate filtering rules.

Limited evaluation of generalization. All five experimental tasks belong to the space-physics domain, and the proposed method explicitly incorporates domain-specific scientific constraints. Therefore, it remains unclear whether the framework would be equally effective on general symbolic regression benchmarks or scientific problems where rich domain-specific priors are unavailable.

Insufficient analysis of computational overhead. The multiple agentic components and search-state management mechanisms may introduce additional inference costs. Comparing only the number of LLM calls is insufficient to fully establish the computational efficiency of the proposed approach.

Questions for authors

1, When the same physical constraints are applied to all methods, how much additional performance improvement does CoTraS provide over conventional search strategies?

2, Could the authors provide additional evaluations on general symbolic regression benchmarks, along with comparisons of end-to-end computational costs and token consumption?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 4YxP

We thank the reviewer for the thoughtful assessment and helpful suggestions. We have added experiments and analyses to better isolate CoTraS’s contribution, evaluate generalization beyond space physics, and examine runtime and token costs. Below, we address your questions and the corresponding concerns in turn.

Q1: When the same physical constraints are applied to all methods, how much additional performance improvement does CoTraS provide over conventional search strategies?

A1: Thank you for raising this important question. In the original experiments, the baseline methods were provided with the same task-specific physical priors as EvoTrace. We will make this shared-prior setting more explicit in the manuscript.

The controlled comparison between EvoTrace and w/o CoTraS, with MA-HASE retained, directly measures the additional benefit of trajectory-guided search. For example, on sunspot forecasting, adding CoTraS reduces MSE from 3998.17 to 1422.16, a 64.4% reduction. On the coupled Fe X task, it reduces RatioMSE from 0.14650 to 0.012333, a 91.6% reduction, despite the ablated version attaining lower individual line-wise errors. This indicates that CoTraS guides search toward modifications that better preserve the coupled relationship, beyond improving isolated fits. These results appear in Appendix D.4, Table 15.

Q2: Could you provide additional evaluations on general symbolic regression benchmarks, along with comparisons of end-to-end computational costs and token consumption?

A2: Thank you for suggesting these additional evaluations. We extended the evaluation beyond space physics to the Biology and Materials subsets of LSR-Synth from LLM-SRBench, covering 24 and 25 problems, respectively. Each problem contains 4,000 training examples, 500 in-distribution (ID) test examples, and 500 out-of-distribution (OOD) test examples. Domain-specific mechanistic constraints are inactive. We report problem-level success rates at 1% and 0.1% relative-error tolerances, together with average time and token consumption. Overall aggregates passing counts across all 49 problems.

MethodAll ACC₀.₀₁All ACC₀.₀₀₁Mat. ACC₀.₀₁Mat. ACC₀.₀₀₁Bio. ACC₀.₀₁Bio. ACC₀.₀₀₁TimeTokens
Qwen3.5-ID
LLM-SR8.160.0012.000.004.170.00378.491.858
LaSR38.7816.3364.0024.0012.508.33203.470.978
SR-Scientist59.1834.6972.0052.0045.8316.6788.264.969
EvoTrace79.5955.1072.0060.0087.5050.0091.252.043
Qwen3.5-OOD
LLM-SR30.6112.2452.0020.008.334.17378.491.858
LaSR48.9828.5784.0048.0012.508.33203.470.978
SR-Scientist55.1040.8276.0064.0033.3316.6788.264.969
EvoTrace61.2242.8676.0064.0045.8320.8391.252.043
Qwen3.6-ID
LLM-SR16.334.0828.008.004.170.00419.901.474
SR-Scientist63.2744.9076.0060.0050.0029.1762.235.449
EvoTrace71.4355.1080.0068.0062.5041.67127.551.863
Qwen3.6-OOD
LLM-SR26.5316.3348.0028.004.174.17419.901.474
SR-Scientist61.2246.9476.0068.0045.8325.0062.235.449
EvoTrace65.3153.0676.0068.0054.1737.50127.551.863

Notes. All = Overall; Mat. = Materials; Bio. = Biology. ACC values are percentages; the subscripts 0.01 and 0.001 indicate 1% and 0.1% relative-error tolerances. Bold marks the best accuracy within each backbone/split block, including ties. Time is minutes per problem; Tokens is recorded input-plus-output usage across model roles, in millions per problem.

EvoTrace achieves the highest Overall accuracy in all eight backbone/split/tolerance comparisons, outperforming the available LLM-SR, LaSR, and SR-Scientist baselines. Under Qwen3.5, ID accuracy improves over SR-Scientist by 20.41 percentage points at both tolerances. The ID/OOD results support its applicability beyond space physics.

As shown in the table above, the recorded cost statistics also reveal complementary resource advantages. EvoTrace takes substantially less time than LLM-SR under both backbones and LaSR under Qwen3.5, while using 58.9% and 65.8% fewer tokens than SR-Scientist under Qwen3.5 and Qwen3.6, respectively. Together, the accuracy results and reference cost statistics highlight EvoTrace’s strong predictive performance and favorable accuracy–resource efficiency.

These results are included in Appendix D.7, Table 19, with the experimental and accounting details in Appendix C.

Reviewer_EX5FOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

When large language models (LLMs) try to discover scientific formulas from data, they often only remember which formula scores high but do not remember "which item was changed to improve it" and "which directions have already failed." To solve this, this paper proposes EvoTrace, an approach for guiding LLMs to perform symbolic regression (SR) via modification of pairing formulas, memory of positive and negative trajectories, and evidence-supported mechanistic constraints. The paper conducts extensive empirical studies and justifies the effectiveness of the proposed method. Results on five space physics tasks and three LLMs show that the system has advantages in multiple predictions and physical diagnostics.

Critical strengths
  • The proposed method is well-motivated by the shortcomings of the conventional LLM-based SR methods, which only remember instead of gaining insights from the previous search.

  • EvoTrace keeps the evidence before and after modifications, along with the history of failures, which helps avoid repeatedly experimenting based solely on the current score.

  • The evaluation includes prediction errors, extrapolation, and task-related physical diagnostics and can demonstrate the difference between fit and structural plausibility.

Critical weaknesses
  • Despite the clear motivation, EvoTrace seems to incorporate additional reflection components into the conventional SR workflow (Figure 1). In addition, the authors claim that EvoTrace is an agentic SR framework, but the paper does not involve any tools that a typical agentic framework would have.

  • The writing, especially the notations, is quite hard to understand. Figure 2 also has some mismatches with the corresponding equations, like Eqs. 3-4 in Figure 2(a). It would be better to have a preliminary section before the method part to summarize and unify the notation to be used, instead of introducing it separately. The figures also seem to be compressed, such as Figures 1, 3, and 4. These figures seem to have an abnormal aspect ratio.

  • The ablation simultaneously removes multiple types of historical and source information, making it difficult to identify the contributions of paired modification evidence, failed families, and mechanism constraints. The activation confidence, support weight, and several contract-fixed parameters of the mechanism relationship still lack complete operational definition and actual configuration.

Questions for authors
  1. Could you provide a discussion or empirical comparison with other frameworks like Pi or open code on these tasks? As they are also agentic frameworks and capable of SR tasks.

  2. Could you provide a common history summary versus a comparison of paired positive and negative modification memories, separate failure families and mechanistic relationships, and provide run-by-run results for ablation?

  3. Could you provide a discussion of the difference between CoTraS and the paper that involves trajectory comparison, like ReasoningBank [1]?

[1] ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. In ICLR, 2026.

5 comments
AuthorsOpenReview ↗

Response to Reviewer EX5F (Part 1)

We sincerely thank the reviewer for the careful reading and constructive feedback. We appreciate your recognition of the clear motivation of EvoTrace. Your comments have helped us clarify the method and strengthen its presentation and evidence. Below, we address W1, W2, and Q1–Q3 in order.

W1: EvoTrace appears to add reflection to a conventional SR workflow, and its agentic tools are not evident.

A1: Thank you for raising this concern. We agree that the agent–tool interaction and the methodological contribution should be explained more clearly.

First, EvoTrace implements an executable agent–tool feedback loop. The Parent Agent accesses two core tools through structured interfaces: data_analyzer executes model-generated Python code on training data and returns numerical summaries or execution errors; equation_evaluator accepts a proposed equation, optimizes its constants, and returns fit metrics, fitted parameters, and numerical-validity feedback. The agent selects tool calls and their arguments, observes the results, and uses the updated search state to decide what to analyze or evaluate next.

Second, EvoTrace connects evaluated modifications to subsequent search decisions. It records what changed from a reference formula and whether that change improved the result under matched evaluation conditions. Supported edits guide subsequent proposals, while recurring failures are archived by formula family to discourage repeated exploration. Candidate retention additionally depends on scientific admissibility, so a lower fitting error alone does not guarantee acceptance. The contribution is this connection between evaluated edits, future search directions, and candidate acceptance. The fine-grained memory comparisons are reported in Appendix D.5.

W2: The notation is difficult to follow, Figure 2 contains inconsistencies with the equations, and several figures appear to have distorted aspect ratios.

A2: Thank you for these helpful suggestions. We have added a preliminary section and notation table before the method, corrected Fig. 2(a) to distinguish utility computation in Eq. (3) from transition labelling in Eq. (4), and restored the original aspect ratios of Figs. 1, 3, and 4. The main additions and revisions are highlighted in blue.

Q1: Could you provide a discussion or empirical comparison with frameworks such as Pi or OpenCode on these tasks?

A1: Thank you for suggesting this comparison. We conducted additional experiments with Pi, a representative general-purpose coding-agent framework, on the five space-physics tasks. We connected Pi to numerical analysis and equation-evaluation tools while retaining its native agent behavior. Both Pi and EvoTrace use Qwen3.5-35B-A3B.

For the tasks in our experiments, both frameworks use identical training and test datasets and the same offline scoring implementation. Each framework is allocated a total task budget of 325 model calls and 325 expression-evaluation attempts; this budget is adaptively allocated across individual tasks by the agent. We ran three independent replicate comparison experiments, with the averaged results shown in the table below.

As shown in the table, EvoTrace achieves lower primary prediction errors across all five tasks and higher R2R^2 in all four columns where it is reported. Across the Fe X and pressure comparisons, its average token consumption is 49.9% lower. These observations support the practical value of EvoTrace’s SR-specific search organization beyond the use of a general-purpose tool-using agent. We will release the EvoTrace implementation, Pi adaptation, experiment configurations, and evaluation scripts upon acceptance.

Method\mathbf{Method}Solar rotation\mathbf{Solar\ rotation}Sunspot\mathbf{Sunspot}Lunar tide\mathbf{Lunar\ tide}Pressure\mathbf{Pressure}Fe X ratio↓\mathbf{Fe\ X\ ratio}\downarrowJoint AST↓\mathbf{Joint\ AST}\downarrowTokens (M)↓\mathbf{Tokens\ (M)}\downarrow
Pi9.472×10−5/0.99789.472\times10^{-5}/0.99784607.74/0.02534607.74/0.02530.6434/0.53360.6434/0.53360.7878/0.70200.7878/0.70200.145700.1457074746.3066.306
EvoTrace2.260×10−5/0.9995\mathbf{2.260\times10^{-5}/0.9995}1422.20/0.6992\mathbf{1422.20/0.6992}0.2406/0.8256\mathbf{0.2406/0.8256}0.5409/0.7954\mathbf{0.5409/0.7954}0.04899\mathbf{0.04899}55\mathbf{55}3.161\mathbf{3.161}

Notes. The first four task columns report MSE↓/R2↑\mathrm{MSE}\downarrow/R^2\uparrow. Bold indicates the better result. Joint AST is the summed complexity of the two Fe X formulas. Tokens (M) reports mean recorded input-plus-output tokens over the Fe X and pressure tasks, in millions.

AuthorsOpenReview ↗

Response to Reviewer EX5F (Part 2)

Q2: Could you compare ordinary history summaries with paired positive and negative modification memories, separate failed-family memory and mechanistic relationships, and provide run-by-run ablation results?

A2: Thank you for suggesting this more fine-grained comparison. We added experiments distinguishing ordinary search history (H), paired modification memory (P), and independent failed-family memory (F). H retains candidate expressions and their observed outcomes. P additionally preserves the reference expression, structural modification, and evaluated outcome, with positive or negative labels reflecting the measured effect. F separately organizes unsuccessful formula families for subsequent retrieval. Domain-specific mechanistic constraints are inactive in these experiments.

We evaluate these configurations on the Biology and Materials subsets of LSR-Synth from LLM-SRBench, covering 24 and 25 problems, respectively. Each problem provides 4,000 training examples and separate ID and OOD test sets of 500 examples each. We report problem-level ACC₀.₀₁ and ACC₀.₀₀₁: the percentage of problems solved at relative-error tolerances of 1% and 0.1%. the percentage of problems solved at relative-error tolerances of 1% and 0.1%. Under the strict scoring rule, a problem is solved when more than 95% of its test points satisfy the tolerance.

Memory\mathbf{Memory}All ACC0.01\mathbf{All\ ACC}_{0.01}All ACC0.001\mathbf{All\ ACC}_{0.001}Mat. ACC0.01\mathbf{Mat.\ ACC}_{0.01}Mat. ACC0.001\mathbf{Mat.\ ACC}_{0.001}Bio. ACC0.01\mathbf{Bio.\ ACC}_{0.01}Bio. ACC0.001\mathbf{Bio.\ ACC}_{0.001}
Qwen3.5−ID\mathbf{Qwen3.5-ID}
H53.0653.0622.4522.4564.0064.0032.0032.0041.6741.6712.5012.50
H+F (w/o P)51.0251.0230.61\mathbf{30.61}64.0064.0040.00\mathbf{40.00}37.5037.5020.8320.83
H+P (w/o F)63.27\mathbf{63.27}30.61\mathbf{30.61}72.00\mathbf{72.00}36.0036.0054.17\mathbf{54.17}25.00\mathbf{25.00}
Qwen3.5−OOD\mathbf{Qwen3.5-OOD}
H55.1055.1034.6934.6964.0064.0052.0052.0045.8345.8316.6716.67
H+F (w/o P)59.1859.1840.82\mathbf{40.82}72.00\mathbf{72.00}60.0060.0045.8345.8320.83\mathbf{20.83}
H+P (w/o F)63.27\mathbf{63.27}40.82\mathbf{40.82}72.00\mathbf{72.00}64.00\mathbf{64.00}54.17\mathbf{54.17}16.6716.67
Qwen3.6−ID\mathbf{Qwen3.6-ID}
H57.1457.1442.8642.8664.0064.0064.00\mathbf{64.00}50.0050.0020.8320.83
H+F (w/o P)57.1457.1434.6934.6964.0064.0044.0044.0050.0050.0025.0025.00
H+P (w/o F)63.27\mathbf{63.27}48.98\mathbf{48.98}72.00\mathbf{72.00}60.0060.0054.17\mathbf{54.17}37.50\mathbf{37.50}
Qwen3.6−OOD\mathbf{Qwen3.6-OOD}
H55.1055.1038.7838.7868.0068.0056.0056.0041.6741.6720.8320.83
H+F (w/o P)59.1859.1840.8240.8272.0072.0056.0056.0045.8345.8325.0025.00
H+P (w/o F)63.27\mathbf{63.27}44.90\mathbf{44.90}76.00\mathbf{76.00}60.00\mathbf{60.00}50.00\mathbf{50.00}29.17\mathbf{29.17}

Notes. All values are percentages; higher is better. All = Overall; Mat. = Materials; Bio. = Biology. Bold indicates the best value within each backbone/split block, including ties.

Adding P improves all eight Overall accuracy entries relative to H. Under Qwen3.5, the ID gains are 10.2010.20 and 8.168.16 percentage points at the two tolerances; under Qwen3.6, they are 6.126.12 and 6.126.12 points. Improvements persist on OOD data. H+P also matches or exceeds H+F in all eight Overall entries, with six improvements and two ties. These results support retaining the measured consequences of structural modifications beyond ordinary history or an independent failure archive.

Definitions and results appear in Appendix C.2 and D.5, Tables 16–17.

Q3: Could you discuss the differences between CoTraS and trajectory-comparison methods such as ReasoningBank?

A3: Thank you for drawing our attention to this closely related work. We have added a discussion in Appendix E acknowledging the connection to ReasoningBank and clarifying the differences. ReasoningBank learns reusable reasoning strategies from successful and failed experiences, and MaTTS also contrasts multiple trajectories. CoTraS focuses on the evaluated consequences of structural formula modifications: it retains the reference expression, the edit, and the revised expression together with their evaluation results under compatible conditions. For example, when a harmonic term is added, the record preserves whether it improves the fit and how it affects complexity and numerical validity.

These reference–edit–outcome records guide subsequent proposals by identifying modifications worth extending and recurring failed formula families to avoid. MA-HASE additionally connects this guidance to scientific admissibility and candidate selection. Whereas ReasoningBank emphasizes retrieving transferable reasoning strategies, CoTraS couples paired structural-modification evidence with iterative control of an ongoing symbolic-regression search. The added discussion is highlighted in blue.

Thank you for your detailed responses and additional experiments.

  • In terms of technical concerns (Q1, Q2), the new results have addressed my concerns. Thanks for the efforts of the authors. A quick follow-up: could you specify the configuration of the Pi coding agent you employ? Do they have a similar budget to EvoTrace?

  • In terms of presentation and soundness concerns (W1, W2, Q3), it seems that the paper has not been updated. We may have further discussion on these concerns once the authors have updated their revised paper.

Overall, I would be happy to raise my score once my concerns are addressed.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Updated Manuscript

We sincerely thank all reviewers for their careful reading and constructive suggestions. We have uploaded a revised manuscript, with the main additions and revisions highlighted in blue for ease of review. The principal updates are summarized below.

  1. Notation and presentation. We added Section 3, Preliminaries and Notation, and Table 1 to consolidate the principal symbols. We corrected the correspondence between Fig. 2(a) and Eqs. (3)–(4), and restored the original aspect ratios of Figs. 1, 3, and 4.
  2. Agent–tool implementation. We expanded Appendix C.2 to describe EvoTrace’s executable data-analysis and equation-evaluation tools, their returned feedback, and how the agent uses these observations to determine subsequent actions. We linked this explanation to the recorded lunar-tide search trajectory in Appendix B.6.
  3. Fine-grained memory comparisons. We added controlled comparisons of ordinary history, paired modification memory, and independent failed-family memory in Appendix D.5, Tables 16–17, covering 49 LSR-Synth problems under two Qwen backbones. Adding paired modification memory improves all eight Overall accuracy entries relative to ordinary history.
  4. Comparison with Pi. We added the Pi comparison in Appendix D.6, Table 18, and specified its software version, backbone, tool interfaces, and shared-budget protocol in Appendix C. EvoTrace achieves lower errors in the four predictive task columns, improved Fe X ratio recovery, and smaller joint expression complexity.
  5. Evaluation beyond space physics. We added evaluations on 49 Biology and Materials problems from LLM-SRBench’s LSR-Synth in Appendix D.7, Table 19. Without activating domain-specific mechanistic constraints, EvoTrace achieves the highest Overall accuracy across both backbones, both ID/OOD splits, and both error tolerances.
  6. Computational costs and reproducibility. We expanded Appendix C to specify the additional data splits, baseline implementations, memory configurations, inference settings, budget accounting, and scoring protocols. We added token-consumption and search-to-freeze time statistics alongside the predictive results, distinguishing matched budget ceilings from actual resource consumption.
  7. Connection to ReasoningBank. We expanded Appendix E to acknowledge the connection to ReasoningBank and MaTTS and clarify how CoTraS organizes paired structural-modification evidence and incorporates it into subsequent search guidance and scientific admissibility.

We thank the reviewers again for helping us improve the paper’s clarity, methodological precision, and experimental evidence. We welcome any further questions.

Kind regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.