acceptodds
Under review as a conference paper at ICLR 2027

Coupled Harness–Model Evolution for Scientific Equation Discovery

Abstract

Large language models (LLMs) are increasingly used to discover scientific equations from experimental data. Such a discovery system has two parts that must work together: a library of executable computational tools and the model's tool-use policy. The tools supply the solving capability, from diagnosing structure in the data to fitting candidate equations; the policy plans and orchestrates their use. Yet existing systems adapt only one part, keeping either the toolset or the model fixed, which leaves the corresponding failures uncorrectable. We develop a coupled evolution framework that improves both parts from the same verifiable fit-error signal. In the tool-evolution phase, an agent grows an executable tool library from a small set of generic numerical operations by diagnosing its own failures: it authors a new tool when a capability is missing and revises its selection rules when the right tool went unused. In the policy-evolution phase, a small open model (roughly 3B active parameters) is fine-tuned on successful tool-use trajectories collected with the frozen library, on training formulas screened to be disjoint from all evaluation benchmarks. Ablations show the two phases are complementary: with no tool library at all the same model solves almost nothing, the tools account for most of the gain, and trajectory training adds a further gain that is largest on the most difficult problems. The full system solves 79.1/71.3% of LSR-Synth problems at fit tolerances /, reaches 63.8% pooled accuracy on the distributionally different SRSD-Feynman panel, and obtains a normalized CP3-28 aggregate of . These results suggest a broader recipe for scientific agents: let the model and its instruments improve each other under verifiable feedback.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Co-Evolution, Agentic LLMs, Scientific Equation Discovery, Symbolic Regression

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_g6iuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a two-phase framework for LLM-based scientific equation discovery that adapts both the tool library and the model’s tool-use policy using the same fit-error signal. In Phase A (harness evolution), GPT-5.5 solves problems from LLM-SRBench with an initial library of 25 generic numerical tools. A second GPT-5.5 instance classifies each failure as a missing capability (it then writes a new tool) or a misused one (it revises the tool interface or selection rule), and an edit is accepted only if all previously solved problems remain solved. In Phase B, the evolved 33-tool harness is frozen and Qwen3.6-35B-A3B is fine-tuned with LoRA on GPT-5.5 trajectories from synthetic tasks screened for overlap with the evaluation formulas. The system is evaluated on LSR-Synth, LSR-Transform, SRSD-Feynman, and CP3-28, and reports higher accuracy than published baselines. Ablations attribute most of the gain to the evolved library.

Critical strengths

Simple, practical recipe. Phase B is plain supervised fine-tuning, with LoRA on about 72 tasks for 120 steps. At deployment the system needs only a frozen 33-tool library and a ~3B-active open model, with no frontier-model calls at inference. That makes it cheap to run and straightforward to reproduce once the library and adapter are released.

Critical weaknesses

Limited novelty. The method combines two existing ideas, harness/tool evolution and trajectory fine-tuning, in a single sequential pass with no feedback between phases. Both have direct precedent in SR (SR-Scientist and KeplerAgent for tool use; A-SR and DecAEvolve for model adaptation). Table 3 also shows that the evolved library without any model nearly matches the full system. Saturating metrics. Under trimmed-5% scoring, LSR-Synth accuracy reaches 96.1/95.4 (Acc@10⁻²/10⁻³), leaving little room to separate methods. Results at stricter tolerances, together with NMSE and symbolic accuracy (both standard in LLM-SRBench), would make the comparison more informative. Missing baselines. Closely related SR systems the paper itself cites, DecAEvolve and KeplerAgent, are not compared. Neither are general evolutionary frameworks such as AlphaEvolve (via OpenEvolve) or ThetaEvolve applied to the same panels.

Questions for authors

How sensitive is Phase A to the initial toolset? The 25 seed tools already reach 41.1/30.2 on LSR-Synth under deterministic traversal (Table 3), roughly matching SR-Scientist’s 63.6 under trimmed scoring (Fig. 3). Would a smaller or differently composed seed set (e.g., only basic fitting and verification) lead to a comparable final library and accuracy?

4 comments
AuthorsOpenReview ↗

Response 1/4

Reviewer's concern:

"Limited novelty. The method combines two existing ideas, harness/tool evolution and trajectory fine-tuning, in a single sequential pass with no feedback between phases. Both have direct precedent in SR (SR-Scientist and KeplerAgent for tool use; A-SR and DecAEvolve for model adaptation)."

We would like to clarify that our work addresses a fundamental methodological bottleneck in the domain: existing approaches evolve either the toolset OR the model, but not both.

Trained agents (SR-Scientist) train models on fixed toolsets. Self-evolving search (A-SR) adapts the search process within a run but does not create or modify executable tools. Symbolic evolution (DecAEvolve) evolves expressions, not executable search procedures. Fixed orchestration (KeplerAgent) provides no tool evolution and relies on physics priors.

However, one-sided adaptation creates an uncorrectable bottleneck when the problem lies on the other side. For example, in trained-model methods, if the hand-made toolset itself is inadequate or encounters new problem types, no amount of training can compensate.

Our contribution is coupled evolution of tools and policy. This represents methodological progress. Unlike previous methods that explore model potential via reinforcement learning on fixed toolsets, our approach enables both components to evolve. The framework introduces (1) failure-driven tool evolution (Algorithm 1, Phase A), (2) trajectory distillation that trains a smaller model to effectively organize evolved tools (Phase B), and (3) achieves 79.1%/71.3% on LSR-Synth-129 and median NMSE 0.000186 on SRSD-240 across diverse domains (Tables 1-2).

We will clarify these distinctions in the revised Related Work (Section 2) and add a comparison table in the appendix.

AuthorsOpenReview ↗

Response 2/4

Reviewer's concern:

"Saturating metrics. Under trimmed-5% scoring, LSR-Synth accuracy reaches 96.1/95.4 (Acc@10⁻²/10⁻³), leaving little room to separate methods. Results at stricter tolerances, together with NMSE and symbolic accuracy (both standard in LLM-SRBench), would make the comparison more informative."

We would like to clarify that the 5% pruning convention is used only for comparison with historical baselines. To directly address the ceiling effect concern, we also report stricter all-points metrics that demonstrate clear room for differentiation. Under full-sample evaluation, our method reaches 71.3% at Acc@10−310^{-3} on LSR-Synth, while the best published baseline (SR-Scientist) reaches about 50% even under the more lenient trimmed protocol. There is still clear room for differentiation.

The differentiation is even more evident on harder benchmarks. On SRSD-240, our method achieves 58.0% on hard tiers versus 44.6% for uDSR. The multi-tolerance breakdown is Acc@10−110^{-1} = 59.6%, Acc@10−210^{-2} = 55.4%, and Acc@10−310^{-3} = 52.1%. The trained model also achieves 33% lower median NMSE than the tools-only baseline (0.000186 vs. 0.000278), showing a clear improvement from policy training.

We provide complete NMSE, multi-tolerance, and symbolic accuracy results below.

Trained Model + Tools (240/240 tasks):

  • Median test NMSE: 0.000186
  • Median test R2R^2: 0.9998

Multi-tolerance breakdown:

  • Acc@10−110^{-1} (τ\tau=0.1): 143/240 = 59.6%
  • Acc@10−210^{-2} (τ\tau=0.01): 133/240 = 55.4%
  • Acc@10−310^{-3} (τ\tau=0.001): 125/240 = 52.1%

T+ baseline: Median NMSE 0.000278, Acc@10−310^{-3}=51.3%. Trained model achieves 33% lower median NMSE.

Following LLM-SRBench's equivalence judgment protocol, we evaluated symbolic accuracy on all tasks using GPT-6.1-sol. Symbolic accuracy is a more demanding metric: it requires the expression to match the ground truth in structure, not only in numerical behavior.

DatasetSymbolic AccuracyNumerical Acc@10−310^{-3}
LSR-Synth-12928.68% (37/129)71.3%
LSRT-11144.14% (49/111)63.1%
SRSD-24031.25% (75/240)52.1%

Lower symbolic accuracy is expected because the criterion is stricter. For example, our predicted expression −0.000123x1+0.316x1e−0.0541x0+0.318x12/(9.91x1+1)-0.000123 x_1 + 0.316 x_1 e^{-0.0541 x_0} + 0.318 x_1^2/(9.91 x_1+1) achieves NMSE = 5.5×10−155.5\times10^{-15} and R2>0.99999999999999R^2 > 0.99999999999999 against the ground truth 0.316Pe−0.0541t+0.316P2/(9.87P+1)0.316 P e^{-0.0541 t} + 0.316 P^2/(9.87 P+1), yet the LLM judges the two as symbolically non-equivalent because of the additional linear term.

We will add these complete multi-tolerance, NMSE, and symbolic accuracy results to the revised manuscript.

AuthorsOpenReview ↗

Response 3/4

Reviewer's concern:

"Missing baselines. Closely related SR systems the paper itself cites, DecAEvolve and KeplerAgent, are not compared. Neither are general evolutionary frameworks such as AlphaEvolve (via OpenEvolve) or ThetaEvolve applied to the same panels."

We would like to clarify that direct comparison is not feasible because these methods evolve fundamentally different objects.

AlphaEvolve / OpenEvolve / ThetaEvolve evolve the "solution" itself. They are designed for per-problem search: given a problem and a scoring function (e.g., accuracy), they evolve and select candidates. Each problem requires running the full evolution process (including LLM sampling) from scratch. DecAEvolve evolves symbolic expressions through test-time RL adaptation and term decomposition. Each problem system triggers per-system RL fine-tuning (GRPO/LoRA), making it a per-problem adaptive method rather than a reusable capability builder. The LLM adapts to each specific system during inference. Test-time cost is therefore high (evolution itself is inference-time search), making them unsuitable for symbolic regression tasks.

Our method evolves the "harness,". The evolution target is the executable tool library (harness): 33 tools optimized via failure-driven diagnosis. Evolution occurs only during training. "Solving" occurs at test time: a small model orchestrates the evolved tools to fit equations. The evolution product is a reusable capability asset across problems, not a specific solution.

Here is a comparison of what each method evolves and when:

MethodEvolution TargetModel TrainingTool EvolutionEvolution Timing
OpenEvolveCandidate programs✗✗Inference
ThetaEvolveProgram generation strategy✓✗Training + Inference
DecAEvolveHypothesis generation strategy✓ (Test-time)✗Inference
OursTool library + Strategy✓ (LoRA)✓Training (Coupled)

For KeplerAgent, we thank the reviewer for the suggestion. Since they have not released their code or reported results on additional datasets, we provide experimental comparison on the LSR-Transform dataset:

MethodMean NMSE
Tool-only (ours)0.496
Trained model + tools (ours)0.121
KeplerAgent @11.50
KeplerAgent @31.21

Our trained system reaches a mean NMSE roughly an order of magnitude lower than KeplerAgent at both @1 and @3.

We will add to the revised manuscript: (1) Expanded Related Work clarifying that these methods address different evolution targets, (2) KeplerAgent comparison on LSR-Transform.

AuthorsOpenReview ↗

Response 4/4

Reviewer's question:

"Phase A starts with 25 seed tools achieving 41.1%/30.2%. How sensitive is the final outcome to initialization? Would a smaller or differently composed seed (e.g., only basic fitting and validation) converge to comparable capability?"

We would like to clarify that the 25-tool seed encodes generic SR knowledge by design. The initial library reflects standard SR operations: basic fitters, diagnostics, verification, structure exploration. This is intentional: evolution's value is measured against a strong baseline, not a trivial one.

Would a smaller seed converge to similar capability? Even if convergence were possible, it would be much slower and lack consistency. We attempted starting with only 6 basic tools. However, facing minimal input information, the tool attribution API pointed to 3 seemingly plausible but distinct fixes across 3 invocations, and none of them generalized to the neighboring 10 cases.

This does not affect our core performance or claims. Any evolution scheme has its operational boundaries within which it demonstrates effectiveness and stability. As observed in A-SR as well, effective iterative self-improvement only occurs in moderate difficulty regimes. Running lengthy and complete/unstable self-evolution for different initial set sizes is extremely expensive and time-consuming, and such boundaries will shift significantly with model capability improvements. While interesting, exploring these boundaries is beyond the scope of this work. Therefore, to better address the scientific problem, we leverage human scientific knowledge to manually design and create the initial tools.

We appreciate the reviewer's constructive feedback, which has helped us better articulate the strengths and clarify the positioning of our work. The comments have been invaluable in highlighting key distinctions and ensuring the paper emphasizes its core contributions more clearly. We will incorporate all the above clarifications and additional experimental details into the revised manuscript and will release the code to facilitate reproduction and further research.

Reviewer_tPcDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a two-phase framework for LLM-based equation discovery. In Phase A, GPT-5.5 grows a library of fitting and analysis tools by diagnosing failed attempts as missing capabilities or tool misuse, with a check that keeps previously solved problems solved. In Phase B, a small open model (Qwen3.6-35B-A3B) is fine-tuned with LoRA on GPT-5.5 trajectories over formula-disjoint synthetic tasks, using the frozen library. Results are reported on LSR-Synth, LSR-Transform, SRSD-Feynman and CP3-28, with ablations separating the library from the trained model.

Critical strengths
  • The evolved tool library seems to work well. With no model in the loop, the evolved library plus BIC selection already outperforms prior LLM-based systems on LSR-Synth. The growth analysis ties the gains to newly added function families. This might be a useful message for LLM-based SR: persistent, verified numerical tools carry a lot of the performance.

  • The tool-growing protocol is sensible and reusable. Failures are split into missing versus misused capability, new tools must pass a held-out gate, and edits are accepted only if no solved problem regresses. This guards against the library drift common in tool-making methods.

Critical weaknesses
  • The coupling claim and the role of the trained model are not well supported. The two phases run once in sequence, with no feedback from the controller to the harness. In the ablations, the model-free traversal with BIC is within about 1.6 points of the full system on LSR-Synth and identical on LSRT111. The final selection also keeps the traversal candidate unless the model’s candidate wins by a margin. The work reads more as an evolved toolbox plus distillation than as co-evolution.

  • The transfer evidence is weaker than presented. The tools and the selection rule were evolved on LSR-Synth and LSRT111, so results there are not held out. LSR-Transform is built from Feynman equations, which makes it close to SRSD-Feynman, and the overlap firewall only covers the Phase B data. CP3-28 is the only clearly external panel, and its comparison mixes scoring procedures.

  • Equation discovery is evaluated only by numeric fit. There is no symbolic accuracy, complexity, or example of discovered equations.

  • Comparisons and reproducibility are limited. Baseline numbers are copied from papers using backbones from 8B to 480B, and there are no GPT-5.5 baselines, even though GPT-5.5 builds the tools and generates the training data. The tools, prompts, trajectories and compute cost are also not described, which makes it hard to see what the system does at each step or what it costs.

Questions for authors
  1. I was wondering if the authors could report GPT-5.5 with the evolved harness, and ideally also without tools, on SRSD-Feynman and CP3-28. Since GPT-5.5 builds the tools and generates the training data, this would help clarify whether Phase B adds capability or mainly transfers the teacher’s behavior to a cheaper model.

  2. Could you please report how often a model-proposed candidate replaces the traversal candidate in the final selection, and whether tool usage differs between the trained and untrained controller? Given how close the full system is to the model-free traversal, this would help show what the trained model contributes.

  3. I was wondering whether the authors could discuss how well the development data is separated from the evaluation data. In particular, which split VERIFY uses during Phase A, how a library evolved on only one of LSR-Synth or LSRT111 performs on the other, and whether the Feynman-derived LSR-Transform formulas overlap with SRSD-Feynman.

  4. Could you please report symbolic accuracy (as in LLM-SRBench), expression complexity, and a few discovered equations next to the ground truth? Without these, it is hard to tell whether the system recovers the underlying laws or finds accurate but complex fits.

  5. I was wondering if the authors could include a list of the final tools (marking which were authored in Phase A), the prompts, and one example trajectory, and indicate which tools produce the final equations in each domain. This would make the method much easier to understand, and would also clarify how much the domain-specific fitters drive the results.

  6. Could you please share the approximate cost of the pipeline (GPT-5.5 usage in Phase A and for teacher rollouts, training compute, and per-problem inference time) and how it compares with the baselines?

  7. I was wondering whether the authors could discuss the fairness of the baseline comparison. The baselines in Table 1 use very different backbones (from Llama3.1-8B to Qwen3-480B), while the proposed system relies on GPT-5.5 to build the tools and generate the training data. It would help to rerun at least one strong baseline, such as SR-Scientist, with a common backbone, or to discuss how backbone strength affects the comparison. It would also help to clarify how SR-Scientist is characterized: the introduction describes it as keeping the model frozen, while Section 5 and Figure 3 say it trains its model on a fixed toolset. A comparison with DecAEvolve, which also fine-tunes the model, would also help position the training part.

7 comments
AuthorsOpenReview ↗

Response 1/7

Reviewer's concern: "I was wondering if the authors could report GPT-5.5 with the evolved harness, and ideally also without tools, on SRSD-Feynman and CP3-28. Since GPT-5.5 builds the tools and generates the training data, this would help clarify whether Phase B adds capability or mainly transfers the teacher's behavior to a cheaper model."

We thank the reviewer for this comment, which makes the paper clearer. We ran the additional experiments below, and the results show that Phase B provides both capability transfer and cost reduction.

Here is the performance comparison on SRSD-240 (Feynman equations benchmark) and CP3:

ConditionSRSD-240CP3
GPT-5.5 direct (no tools)72.1% (173/240)-4.612
GPT-5.5 + evolved tools59.6% (143/240)-6.688
Untrained 3B + evolved tools59.6% (143/240)-6.688
Trained 3B + evolved tools63.8% (153/240)-7.268

The table shows that Phase B adds capability, not just cost reduction. The one exception is SRSD-240, where GPT-5.5 likely memorized many Feynman equations during pretraining, which explains its high performance without tools on this benchmark. On CP3, where the ground truth is unknown, GPT-5.5 without tools performs worse overall.

Phase B contributes in two ways: (1) It transfers and improves upon the teacher's behavior. The trained 3B model learns to organize the evolved tools more effectively than GPT-5.5 did, achieving +0.580 improvement on CP3-28 and +11.3pp on SRSD-240 compared to GPT-5.5+tools. (2) It reduces inference cost while maintaining or exceeding teacher performance. Both aspects demonstrate that Phase B provides substantial value beyond simple behavior cloning.

We thank the reviewer for this important question. The comparison has helped us clarify that Phase B provides both capability gains and cost efficiency. We will emphasize this finding in the final manuscript.

AuthorsOpenReview ↗

Response 2/7

Reviewer's concern: "Could you please report how often a model-proposed candidate replaces the traversal candidate in the final selection, and whether tool usage differs between the trained and untrained controller? Given how close the full system is to the model-free traversal, this would help show what the trained model contributes."

We examined the corresponding experiment logs. The results demonstrate that the trained model discovers different solutions and improves performance on difficult problems.

The trained model produces different final equations in about 40% of cases. We compared tool-only exhaustive traversal with BIC selection vs trained 3B with BIC over SRSD-240 tasks:

MetricResult
Identical predictions147/240(61.25%)
Different predictions93/240 (38.75%)

This divergence rate demonstrates that the learned controller makes substantially different orchestration choices compared to fixed exhaustive traversal. However, when both methods already achieve accuracy within the 0.01 threshold, further improvements to the discovered equation do not manifest as changes in [email protected].

The comparison between GPT-5.5 controller and trained model showed 13.79 tools/task vs 9.40 tools/task. The tool-only traversal uses a fixed exploration strategy rather than a learned controller. Performance comparison shows the trained model improves over tool-only baseline on held-out benchmarks. The gains are particularly pronounced on hard-tier SRSD problems: 14/50 (28.0%) → 20/50 (40.0%) without BIC (+12pp), and 24/50 (48.0%) → 29/50 (58.0%) with BIC (+10pp).

Also for context, the evolved tool library from Phase A is itself a significant contribution. For few-shot tasks, tools alone can be sufficient. For sustained use cases, training cost amortizes and the improved performance justifies the one-time investment in model distillation.

We thank the reviewer for highlighting the importance of distinguishing between cost savings and quality improvements. We will emphasize these findings in the final manuscript and release the complete evaluation data to support reproducibility.

AuthorsOpenReview ↗

Response 3/7

Reviewer's question: "I was wondering whether the authors could discuss how well the development data is separated from the evaluation data. In particular, which split VERIFY uses during Phase A, how a library evolved on only one of LSR-Synth or LSRT111 performs on the other, and whether the Feynman-derived LSR-Transform formulas overlap with SRSD-Feynman."

We address development/evaluation separation, VERIFY split usage, and LSR-Transform vs SRSD-Feynman overlap.

Q1: VERIFY Split in Phase A (Section 3.2)

VERIFY uses training split only (D_fit, 60% of each benchmark, the fitting subset from Equation 1 in the paper). The interaction interface programmatically blocks access to D_sel (public selection subset, 20%) and D_test (test subset, 20%) during Phase A. This ensures that tool evolution operates only on the training data.

Q2: Cross-Dataset Generalization

We measure structural similarity using Normalized Edit Distance (NED), where NED = 0 indicates identical formulas and NED = 1 indicates completely different structures:

ComparisonMedian NEDHigh Similarity (NED<0.3)
LSR-Synth ↔ LSRT1110.810.6%
LSRT111 ↔ SRSD-2400.731.7%
LSR-Synth ↔ SRSD-2400.791.2%

Formula distributions are structurally distinct. LSR-Transform and SRSD-Feynman both derive from the 120 Feynman equations, but transformations alter structure substantially.

Importantly, the tools encode general SR strategies rather than dataset-specific formulas. Manual inspection confirms that no tool memorizes specific formulas, and all code will be released upon acceptance. The self-evolved library is itself a core contribution of our work, and its strong performance on two held-out datasets (CP3 and SRSD-240) further demonstrates its generalizability.

Reviewer_X7Y1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies a two-stage system for scientific equation discovery. A frontier model first expands and revises an executable numerical toolkit using failure feedback. A smaller Qwen controller is then trained on successful tool-use actions for 72 synthetic tasks, with the toolkit fixed. The system is evaluated on LLM-SRBench and on two separate benchmark families. The central proposal is that adapting both the tools and their controller addresses complementary sources of failure.

Critical strengths
  1. The paper addresses a useful question: whether equation-discovery failures come from missing numerical tools or from the model's inability to use them. The two-stage framework provides a concrete way to improve both components.

  2. The deterministic tool-only baseline is a useful control. Comparing initial and evolved libraries alongside untrained and trained controllers helps identify where the performance gains come from.

  3. The evolved toolkit transfers to SRSD-Feynman and CP3-28, which are excluded from its development. These results provide evidence beyond the main development benchmarks, and the paper clearly acknowledges differences in external scoring conventions.

Critical weaknesses
  1. The main benchmarks are also used to develop the toolkit. LSR-Synth and LSR-Transform supply the Phase-A development stream, so holding out test rows or filtering Phase-B formulas does not establish generalization to unseen tasks. A task-disjoint evaluation is needed to support the headline comparison (Section 4.1).

  2. The additional benefit of the trained controller is modest on the main benchmarks. It improves over evolved deterministic tools by only 1.6/1.5 accuracy points on LSR-Synth and gives no improvement on LSR-Transform (Table 3). Repeated runs and comparable runtime budgets would help establish whether this added complexity is worthwhile.

  3. The proposed evolution mechanisms are not individually validated. Failure classification and monotone acceptance distinguish the approach from simply expanding a toolkit and then distilling its use, but there are no targeted ablations showing which of these design choices matters.

  4. Important implementation details remain unclear. The overlap filter specifies affine equivalence mathematically without explaining how it is checked computationally. Probe counts, numerical tolerances, and candidate-promotion margins should also be reported to make the procedure reproducible (Appendices A.7–A.10).

Questions for authors

See weaknesses.

5 comments
AuthorsOpenReview ↗

Response 1/4

Thank you for your detailed and constructive review. We address each concern below:

Reviewer's concern: "The main benchmarks are also used to develop the toolkit. LSR-Synth and LSR-Transform supply the Phase-A development stream, so holding out test rows or filtering Phase-B formulas does not establish generalization to unseen tasks. A task-disjoint evaluation is needed to support the headline comparison (Section 4.1)."

We understand the concern about information leakage and would like to clarify: the training formulas in both Phase A and Phase B are completely disjoint from all evaluation formulas under three independent equivalence tests, not merely filtering Phase B formulas.

Here are the datasets we used in each phase:

  • Phase A (harness evolution): Uses LSR-Synth and LSR-Transform datasets
  • Phase B (policy training): Uses programmatically synthesized formulas (Section 3.3)
  • Evaluation: Beyond Phase A datasets, we evaluate on held-out benchmarks SRSD-Feynman-240 and CP3-28. Our system achieves state-of-the-art performance on both tool evolution sets and completely unseen benchmarks:
Benchmark CategoryBenchmarkBest PublishedOurs
Tool Evolution SetsLSR-Synth-12963.6 (SR-Scientist)79.1
LSR-Transform-11156.76 (PySR)77.5
Unseen Test SetsSRSD-Feynman-24044.6 (uDSR)63.8
CP3-28-6.20 (QLattice)-7.27

Note: LSR-Synth/Transform use accuracy @0.01; SRSD-Feynman uses accuracy at R2>0.999R^2 > 0.999; CP3-28 uses mean log⁡10\log_{10}NMSE (lower is better).

To ensure complete disjointness, we employ three complementary equivalence tests (detailed in Appendix A.7 and Response 4/4 below). We executed full pairwise collision detection across all benchmark pairs using the actual ground-truth formulas and detected zero collisions. This demonstrates the independence of our training sets (both Phase A and Phase B) from all evaluation sets.

We believe this addresses the generalization concern and demonstrates that our evaluation is on genuinely unseen tasks.

AuthorsOpenReview ↗

Response 2/4

Reviewer's concern: "It improves over evolved deterministic tools by only 1.6/1.5 accuracy points on LSR-Synth and gives no improvement on LSR-Transform. Repeated runs and comparable runtime budgets would help establish whether this added complexity is worthwhile."

We agree with the reviewer's observation and would like to provide clarification: the gains on development sets are expected, while held-out benchmarks show clear value from training.

The reviewer correctly notes limited gains on LSR-Synth and LSR-Transform development sets. This is expected because these datasets participated in Phase A tool evolution—tools are already optimized for this distribution. The tool-only baseline is exceptionally strong (70.5% at [email protected]), already surpassing external baselines. This demonstrates successful tool self-evolution, which is itself a core contribution of this work.

Moreover, on held-out benchmarks (SRSD-Feynman and CP3-28), model + tools demonstrates meaningful improvements over tools-only:

  • SRSD-Feynman-240 hard tier: 58.0% → 63.8%, +5.8pp gain
  • CP3-28 pooled: mean log⁡10\log_{10}NMSE improves by +0.36 magnitude (-6.86 → -7.27)

These gains on unseen distributions validate the training phase's contribution beyond tool evolution.

Phase B training cost: 49 minutes, 3.3 GPU-hours total (4× GPUs, 50 steps, LoRA rank 16)

We appreciate the reviewer's insight and agree that the cost-benefit tradeoff depends on use cases. For few-shot tasks, evolved tools alone (the frozen library from Phase A) are sufficient. For continued use scenarios, training cost amortizes and the +5.8pp and +0.36 magnitude gains on held-out benchmarks justify this investment. We will expand this discussion in the revision to clarify when training is worthwhile.

AuthorsOpenReview ↗

Response 3/4

Reviewer's concern: "The proposed evolution mechanisms are not individually validated. Failure classification and monotone acceptance distinguish the approach from simply expanding a toolkit and then distilling its use, but there are no targeted ablations showing which of these design choices matters"

We appreciate this feedback and would like to clarify the distinct roles of these two mechanisms: failure classification is necessary routing logic, while monotone acceptance provides empirical stability guarantees.

Failure Classification (DIAGNOSE) is a necessary routing decision determining the root cause of failure. Without failure classification, the system cannot determine whether to edit the tool library L\mathcal{L} or the harness H\mathcal{H}. This is routing logic. The alternative would be random editing or always editing both components, neither of which is reasonable.

Monotone Acceptance ensures stability throughout evolution by re-validating on previously solved instances, preventing performance degradation from tool conflicts. As demonstrated in Figure 3, LSR-Synth in-domain accuracy monotonically increases without performance regression. Without this mechanism, we observed frequent performance oscillations where improvements on new tasks caused regressions on previously solved ones.

These harmful modifications are typical examples that monotone acceptance prevents:

(1) tool parameter over-tightening that reduces search capacity (e.g., decreasing max_terms), causing complex tasks to fail;

(2) path deletion or replacement that simplifies tool invocation routing (e.g., removing fallback paths), causing tasks requiring alternative strategies to fail;

(3) threshold over-strictness that increases validation standards (e.g., tightening condition number thresholds), causing boundary-acceptable solutions to be incorrectly rejected.

We will expand the discussion in revision to give this mechanism more prominence.

Comments on the paper

1 comment

We sincerely thank all reviewers and the committee for their thorough examination and constructive feedback. We have addressed every concern raised through clarifications and new experimental evidence, which we integrated into the revised manuscript (highlighted in blue). Through these responses, we demonstrate that our coupled evolution framework significantly advances symbolic regression systems both mechanistically and empirically, achieving state-of-the-art performance across multiple benchmarks.

Regarding Reviewer X7Y1

  1. We have clarified that there is no overlap between our tool evolution dataset and the evaluation dataset, ensuring proper separation of concerns.
  2. We explained why tool and model performance appear similar on the development set, and justified the necessity of training on the held-out evaluation set to demonstrate generalization.
  3. We provided justification for the mechanisms employed in our framework and their necessity for achieving the reported improvements.
  4. We have added all requested experimental details.

Regarding Reviewer tPcD

  1. We report GPT-5.5 performance under identical tool configurations. Our model demonstrates superior tool adaptation, further validating the advantages of coupled training over simply providing tools to frontier models.
  2. We have explicitly highlighted the model's incremental gains relative to the evolved tool library.
  3. We confirmed that our tool evolution dataset and evaluation dataset are completely disjoint, with no data leakage.
  4. We have addressed all raised points including symbolic regression accuracy metrics, complete tool listings, computational cost analysis, and expanded baseline comparisons.

Regarding Reviewer g6iu

  1. We addressed novelty concerns by emphasizing that our approach differs mechanistically from all baselines. Our tool evolution strategy and the resulting performance gains are achievements no prior work has demonstrated.
  2. We clarified that the datasets have not reached their ceiling and substantial room for improvement remains. Some metrics approaching 100% simply reflect competitive performance against existing methods, not task saturation or triviality.
  3. We distinguished our approach from the methods the reviewer mentioned and added comparisons with additional relevant baselines.
  4. We justified our choice of initial evolution set and responded to questions about the feasibility of starting from smaller seed sets.

We believe these comprehensive responses and additional experiments address the reviewers' concerns and strengthen the paper's contribution. The coupled-evolution framework represents a principled advance in symbolic regression, and we are committed to incorporating all clarifications into the camera-ready version. We appreciate the opportunity to revise and hope the committee finds our responses satisfactory.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.