acceptodds
Under review as a conference paper at ICLR 2027

EvoSim: Learning to Model, Modeling to Learn

Abstract

Physics-based models connect scientific explanation with quantitative prediction. Constructing them requires selecting physical processes, defining states and governing equations, specifying couplings, and identifying parameters from experiments. Existing AI systems remain limited in making these model structure decisions autonomously. We introduce EvoSim, a self-evolving AI scientist for physical modeling. It uses experimental discrepancies to drive mechanism and equation revisions and held-out experimental data to test physical plausibility. Exploration traces make updates to knowledge, skills, and multi-agent orchestration. This co-evolution improves physics-based models and EvoSim’s ability to select mechanisms, diagnose failures, and coordinate research. We evaluate EvoSim on two industrial battery modeling tasks. It predicts lithium-metal-plating onset from 25 to 45 °C and 2 C to 6 C with a mean absolute error of 1.79% in state of charge. Dynamic voltage prediction under vehicle driving conditions achieves a root mean square error of 7.62 mV, surpassing the reported accuracy of models developed by human experts. Self-evolution reduces model and physics errors by approximately 36% relative to baseline, demonstrating improved scientific modeling capability. EvoSim turns experimental observations into validated models and cumulative research expertise.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI Scientist, Recursive Self-Improvement, Physical Modeling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6xEvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EvoSim is an AI scientist framework for physics-based modeling. It revises physical models using experimental feedback and turns successful and failed investigations into reusable modeling procedures. The intended contribution is the joint improvement of the physical model and the agent’s modeling capabilities. Experiments cover lithium-plating onset, dynamic-voltage prediction, and the effect of retained procedures on later modeling tasks.

Critical strengths

The problem is important. Constructing and revising physical models is more demanding than simply running an existing simulator. Checking whether proposed mechanisms are actually implemented and supported by measurements is a useful part of the system.

The results suggest practical value. Across 45 matched pairs, model/physics failures decrease from 33 to 21. This provides encouraging evidence that the learned procedures improve subsequent modeling reliability.

Critical weaknesses

The methodological novelty remains unclear. The paper discusses related experience-learning methods, but the experiments do not establish what Scientific Evolution adds beyond a well-designed procedural memory. In Table 12, Naive Memory omits the exact recurrence and implementation checks that Scientific Evolution receives. The comparison therefore shows that actionable procedures outperform this particular summary, rather than establishing an advantage over a strong experience-learning baseline.

The evidence for continuing co-evolution is also limited. Table 19 documents successive updates, but the 45-pair evaluation uses procedures frozen beforehand. These results support procedure reuse, without isolating the benefit of continued updates. Moreover, the main controlled example concerns implementation repair rather than improved physical-mechanism selection. This leaves a gap between the demonstrated benefit and the broader modeling claim.

Questions for authors
  1. Can the authors compare Scientific Evolution with a strong structured-memory baseline built from the same source trajectories, using comparable resources and repeated runs?

  2. Does continuing procedure evolution outperform a frozen procedure library under the same budget? Evidence on a task requiring mechanism or equation selection would be particularly helpful.

5 comments
AuthorsOpenReview ↗

[Part 1/5] Author Response

Thank you for your careful review. Your comments concern four points: the novelty of the method; the comparison with a strong baseline; the role of continued updates; and the evidence on mechanism selection. Parts 1 and 2 state how EvoSim differs from existing methods. Parts 2 and 3 test these differences in a new experiment and compare with strong baselines, including repeated runs. Part 4 addresses continued updates. Part 5 addresses mechanism selection.


Weakness 1: The methodological novelty remains unclear. The paper discusses related experience-learning methods, but the experiments do not establish what Scientific Evolution adds beyond a well-designed procedural memory.

EvoSim is not a procedural memory. It differs from a procedural memory in three respects: what signal it learns from; what condition stored experience must meet to be retained; and what is updated besides the stored content. These three points are the method of Sections 3.2 and 3.3 itself. We explain them one by one below, and then test them one by one in an experiment. Scientific Evolution in the submission is the part of EvoSim that accumulates and updates experience; below we write EvoSim throughout.

EvoSim calls each stored practice a procedure. A procedure states four things: what to do; what effect is expected afterwards (expected effect); under what conditions to use it (conditions of use); and which checks confirm it. Besides procedures, EvoSim stores physics findings, failure experience, tools, checks and research strategies. Section 3.1 calls all of this the scientist state, which is what you refer to as the procedure library; we call it the "procedure library" below.

Table A compares EvoSim with existing methods on the three points above.

Table A. EvoSim and existing methods.

MethodWhat is storedLearning signalCondition for retentionRecords conditions of useWhat is updated
Voyager [4]Executable skill codeWhether the task is completedCurrent task succeedsNoSkill library
ExpeL [2]Text insights and successful examplesSuccess or failure of training tasksLLM adds, edits and votesNoInsight list
AWM [1]Step-by-step workflowsTrajectories judged successfulTrajectory succeedsNoWorkflow memory
ACE [5]Strategy entriesExecution outcomesHelpful and harmful countsNoContext
EvoScientist [7]Ideation and experiment strategiesIdea rankings and code execution resultsDistilled by a manager agentNoTwo memories
DGM [6]Agent source codeBenchmark scoreTotal score of the modified agentn/aAgent code
Huang et al. [8]One skill documentPass rate given by a physics evaluatorHighest pass rate on a held-out validation setNoSkill document
HExA [9]Strategy skills and error recordsTrajectory rewardRanked and pruned by mean rewardYesSkill library
EvoSimFindings, failure experience, procedures, tools, checksDense: evidence from every test of a candidate model (deviation from measurements, implementation checks, physical checks)The procedure shows its expected effect on other tasksYesKnowledge, skills, multi-agent orchestration

The three bold cells are the three differences.

Learning signal. The learning signal of existing methods is sparse: when a task ends there is one outcome, success or failure, and the method learns from that one outcome. In web navigation and games this is reasonable, because tasks are short and the outcome is clear. Scientific modeling deals with frontier problems, unknown mechanisms and complex domains, and aims at models more effective than those of human experts. Such a task is long and hard: one investigation goes through many steps of proposing hypotheses, implementing, solving and testing. Looking only at the final outcome discards the many valuable signals in these steps. The learning signal of EvoSim is dense. Within a task it tests every candidate model, and every test leaves evidence: how far predictions are from measurements; whether the proposed change is actually active in the executed model; whether the result satisfies physical constraints. Rejected candidate models leave evidence too. A model that fits well but does not pass these checks is recorded as a failure in EvoSim.

AuthorsOpenReview ↗

[Part 2/5] Author Response

Weakness 1 (cont.): [...] the experiments do not establish what Scientific Evolution adds beyond a well-designed procedural memory.

Condition for retention. Existing methods retain experience when a task succeeds, or let a language model decide whether to retain it. EvoSim treats each procedure as a testable prediction. A procedure is first tried on other tasks, i.e. tasks that played no part in proposing it. It is retained only if its expected effect actually appears. This is the rule of Section 3.3; we call it the "retention rule" below. DGM also validates changes by experiment, but what it tests is the total score of the whole agent. EvoSim tests each procedure against its own expected effect.

What is updated. Memory methods update only the stored content. DGM modifies the code of the agent but does not accumulate task experience. With the same retention rule EvoSim updates three kinds of content, which the abstract calls knowledge, skills and multi-agent orchestration. Knowledge means physics findings and failure experience. Skills means procedures, tools and checks. Multi-agent orchestration means research strategies and the instructions given to each agent. A research strategy specifies how an investigation is organized: how many candidate branches run in parallel on a task, over how many rounds, and how the final answer is chosen. A "candidate branch" (Section 3.2) is one independent investigation run in parallel on the same task.

[8] and [9] in Table A are two recent preprints that apply skill updating to physics tasks. The skills of HExA also state conditions of use. But both works still retain skills by the pass rate or reward of the task, and update only the skills themselves.


Weakness 2 & Question 1: In Table 12, Naive Memory omits the exact recurrence and implementation checks that Scientific Evolution receives. The comparison therefore shows that actionable procedures outperform this particular summary, rather than establishing an advantage over a strong experience-learning baseline. Can the authors compare Scientific Evolution with a strong structured-memory baseline built from the same source trajectories, using comparable resources and repeated runs?

These two comments are answered by one new experiment. It compares EvoSim with two structured-memory baselines under the same conditions, and removes the three differences above one at a time to see how the result changes. The setup is below; the results are in Part 3.

Baselines. ExpeL [2] stores text insights with importance counts and retrieves similar successful examples. AWM [1] induces step-by-step workflows from successful trajectories. We take AWM as the representative of a well-designed procedural memory.

Task. NewtonBench [3] is a benchmark for physical law discovery. Each task hides a physical law. The law has been deliberately altered, for example in an exponent or a constant, so it cannot be answered by recalling a textbook. The agent collects noisy data through simulated experiments and submits the law it finds. A submission counts as correct only if it is mathematically equivalent to the hidden law. The tasks cover 12 physics domains outside batteries. The law of each domain has 3 altered versions. Each version has 3 observation modes: the output of the law can be observed directly; the law is hidden inside a simple physical system; the law is hidden inside a complex physical system. In the last two modes only the output of the system is observed, so the system must be modeled before the law can be found. We use all tasks of the hard level, 12 × 3 × 3 = 108 in total.

Protocol. There are three steps, the same for the three methods. First, an agent with no experience attempts each of the 108 tasks once and leaves research records. While working on a task the agent does not know whether it is right; after the task ends, each record is labeled correct or wrong. These labeled records are the common source of the three methods. Second, each method builds its own memory from them. The 3 versions of a law are 3 different laws. For a task of one version, a method uses only the records and labels of tasks of the other two versions. The answer to a tested law has therefore not been given to any method before testing. ExpeL and AWM distill as described in their papers. EvoSim builds its procedure library: it proposes procedures from the records, tries them on other tasks, and retains only the procedures whose expected effect appears. Third, testing: each method attempts each of the 108 tasks once with its memory; records made during testing are not labeled. The three methods use the same base model and the same per-task budget. ExpeL and AWM do not update the research strategy; they choose the number of candidate branches themselves, up to a limit. The research strategy that EvoSim uses was obtained by its own updates and is part of the procedure library.

AuthorsOpenReview ↗

[Part 3/5] Author Response

Weakness 2 & Question 1 (cont.): [...] Can the authors compare Scientific Evolution with a strong structured-memory baseline built from the same source trajectories, using comparable resources and repeated runs?

Table B. Number of tasks on which the correct law is found.

RowMethodDirect observation (36)Simple system (36)Complex system (36)Total (108)
1No experience2213944
2ExpeL3419962
3AWM31191060
4EvoSim, library not modified during testing31342186
5EvoSim, library not modified during testing, w/o retention rule28322181
6EvoSim, library not modified during testing, w/o learned research strategy (single candidate branch)29261873
7EvoSim, library updated during testing32332388

Comparison with strong baselines. The memory of ExpeL and AWM does not change during testing. EvoSim in row 4 does not modify its procedure library either, so the conditions are the same. It finds 86 laws; ExpeL finds 62 and AWM 60.

Evidence for the learning signal. When the output of the law can be observed directly, modeling takes few steps, and the three methods are close (EvoSim 31, ExpeL 34, AWM 31). When the law is hidden in a complex system, the system must be modeled first; there are many steps, and looking only at the final outcome discards more. Take the 3 complex-system tasks on Malus's law: all 18 candidate branches fitted the data down to the noise level, and 17 of them had fitted the wrong physical quantity. By final fit error all 17 count as successes, so a method that learns only from success or failure receives no signal here that anything needs correcting. The update of EvoSim reads the test evidence of every candidate model in the research records, not the final outcome. From the records of another domain it obtained a procedure that corrected this error (Example 1 in Part 4). On the 36 complex-system tasks AWM finds 10 laws and ExpeL 9, the same level as the 9 found without experience. EvoSim finds 21, and 23 with the update during testing.

Evidence for the retention condition. Row 5 removes the retention rule: proposed procedures are put into the library without being tried, and everything else is as in row 4. The number of correct laws drops from 86 to 81.

Evidence for what is updated. One of the things EvoSim updates is the research strategy (how the investigation of one task is organized: how many candidate branches run in parallel, over how many rounds, and how the final answer is chosen). Row 4 uses the research strategy that EvoSim learned itself. Row 6 replaces it with the simplest organization: a single candidate branch per task, which receives the whole budget previously shared among the branches; everything else is as in row 4. The number of correct laws drops from 86 to 73. The total budget is unchanged, so these 13 tasks are the gain from the learned research strategy. ExpeL and AWM update only the stored content and have no counterpart to this. The 73 laws of row 6 are still more than those of ExpeL (62) and AWM (60); this part of the gap comes from the first two differences.

Row 7 is the full EvoSim, which also updates the procedure library during testing. It is explained in Part 4.

Repeated runs. In the experiment above, each method attempts 108 different tasks once. In addition, we repeated the experiment of Section 4.3 three times and added ExpeL and AWM. This is the experiment behind Table 12. The task starts from a faulty voltage model: its predicted battery voltage differs from the measurements by about 89 mV. The system has to find the fault and repair the model. ExpeL, AWM and EvoSim use the records of the same earlier investigation. The runs use gpt-5.6-sol on 30 °C DST, with at most 3 solves per run.

Table C. Lowest valid voltage RMSE obtained in each run (mV).

ConditionRun 1Run 2Run 3Runs repaired
Baseline (no experience)89.4889.29394.880/3
Naive Memoryno valid solution89.2089.380/3
AWM89.2983.6489.570/3
ExpeLno valid solution89.3821.561/3
Scientific Evolutionno valid solution21.5621.562/3

Scientific Evolution in the table is EvoSim. It repairs the model in 2 of the 3 runs, reducing the error to 21.56 mV, consistent with the 22.02 mV in the submission. ExpeL repairs it once. AWM, Naive Memory and Baseline never repair it, and their error stays at about 89 mV. The ordering of the methods is the same as in Table B. On this task the procedural memory AWM is no different from the Baseline without experience.

Reviewer_XMnwOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvoSim is an AI workflow for physics-based modelling that builds and revises mechanistic models of a physical system, and that also accumulates the modelling skills it uses along the way. Two processes run together. Model evolution uses discrepancies against measurements to propose mechanism and equation revisions, investigates competing explanations in parallel, and admits a candidate only after it passes implementation, solvability and measurement checks. Scientific evolution reviews the investigation records and retains reusable procedures, each carrying an action, an expected effect, checks and the conditions under which it was supported. The system is evaluated on two battery modelling tasks, both solved in COMSOL against published measurements: lithium-plating onset, where 20 measured conditions from 61 cells are split 14 for development and 6 for validation and the model reaches a mean absolute error of 1.79 SOC percentage points, and dynamic voltage, where 16 A123 DST and US06 traces of 109,746 samples are evaluated by leaving one temperature out and the model reaches 7.62 mV at 30 °C DST and 8.97 mV at 20 °C US06. A further study reports 45 matched pairs across 11 modelling tasks, in which model and mechanism failures fall from 73.3% to 46.7%. The paper runs no new physical experiments; the models are built against existing measured data.

Critical strengths
  1. The evaluation uses real measured data rather than a toy system, and the physics is specified in full. The plating model is built on 20 measured onset conditions from 61 Li-graphite cells, and the voltage study uses 16 complete driving traces totalling 109,746 samples. Equations 7 to 24 and 26 to 44 give the transport, kinetics, plating and terminal-voltage relations with their coefficients, so a reader can check the model instead of taking the numbers on trust. The reported errors are also placed against concrete references: 1.79 SOC points against 3.79 for the constant-fraction representation on the same six held-out conditions, and 7.62 mV against values reported for other dynamic-voltage models.

  2. The retention rule is a principled design, and it is the paper's most interesting idea. A method is not kept because it once worked. It is kept only when applying it produces the expected effect under the recorded conditions, and it is stored together with those conditions and with the checks it must pass (L132, L389–L390). This is deliberately separate from accepting a model, which uses the task's implementation, solution and measurement criteria (L245), so a rejected model can still leave behind a usable method. The criterion matches the predictive core of physical theory: a mechanism should say what will happen, not merely fit what happened.

  3. The reporting is unusually candid. A rejected hypothesis is reported rather than buried: the general solid-electrolyte-interphase mechanism fails in 0 of 3 tested forms and the system rejects it. Table 30 labels each of nine cross-task conclusions as tested only, limited, or unsupported, which is the paper auditing its own claims. The physics interpretation is careful too: the rate-dependent inactivation yield is explained in terms of the time available during charge and the time needed for deposited lithium to lose reversibility, and the paper explicitly notes that a lower irreversible fraction at high rate does not mean high-rate charging plates less.

Critical weaknesses
  1. Reproducibility is weak. There is no code, no pseudocode for the agent loop, and no controller prompt in the appendix, and the submission carries no implementation. The paper does describe the numerical setting, the splits, the solver tolerances and the record fields, so a reader can see what was configured, but none of it can be run. For a system whose claim rests on what its search procedure and its retained procedures do, the procedure itself is the one artefact a reader cannot inspect.

  2. The experiments are too narrow. Two tasks carry the headline results and both are electrochemical models of lithium-ion cells solved in one commercial solver. The 45-pair study widens the task types, covering parameter identification, forward prediction, mechanism comparison and diagnosis, but not the domain: every task family is battery modelling. Nothing in the paper shows whether the procedure transfers to a different physical system, so the claim of general physics modelling rests on one domain.

  3. The paper's framing implies discovery of new model structure, while the evaluated mechanisms are re-derived under new conditions. Lithium plating, electrolyte transport and double-layer charging are established processes, present in the literature the paper itself cites, so what is demonstrated is the construction of a model with these mechanisms and improved parameterisation for particular cells, not the discovery of a mechanism. A counterfactual condition, in the spirit of benchmarks such as NewtonBench, where the governing law is deliberately altered so that recall cannot substitute for inference, would support the stronger claim. As it stands the paper supports the weaker one.

  4. Few ablations isolate the techniques that the paper contributes. Baseline, chronological memory and the procedure version are compared, and the 45-pair study decomposes six failure modes, but no condition isolates the contribution of the retention gate, of parallel investigation, or of the procedure store itself. The gate is the most distinctive of the three and the one with the least evidence: a version that stores procedures without requiring the expected effect to be reproduced would show directly what the gate buys.

  5. Work on skill and memory evolution in agents is not discussed. Voyager keeps an ever-growing library of executable skills, ExpeL extracts natural-language insights from experience, and Agent Workflow Memory induces reusable routines from past trajectories. None is cited. EvoSim's procedures differ from these, since they carry conditions of use and describe how to model rather than what to do, but the difference is left for the reader to infer, and a reader who knows that literature will read the retention mechanism as a familiar idea.

Questions for authors
  1. Can the code, the agent loop and the controller prompts be released, or summarised as pseudocode, so that the search procedure and the retained procedures can be inspected?

  2. Can the evaluation include physical systems beyond lithium-ion cells, so that the transfer of the procedure, rather than its fit to one model family, can be assessed?

  3. Can the paper add one counterfactual condition, in which the governing mechanism is altered rather than taken from the literature, to separate model construction from discovery?

  4. Can the contribution of the retention gate, parallel investigation and the procedure store be ablated separately? For the gate, a variant that retains procedures without requiring the expected effect to be reproduced would be most informative.

  5. Can the related work on agent skill and memory libraries, for example Voyager, ExpeL and Agent Workflow Memory, be discussed, and the difference between those memories and EvoSim's procedures be stated explicitly?

9 comments
AuthorsOpenReview ↗

[Part 1/7] Author Response

Thank you for your review and for your support of the paper. You raised five points, and we answer them in order over seven replies. Parts 1 and 2 address reproducibility and give two pieces of pseudocode. Part 3 addresses physical systems beyond batteries and Part 4 the counterfactual condition, both with one new experiment. Part 5 gives the three ablations. Parts 6 and 7 address related work.


Weakness 1 & Question 1: Reproducibility is weak. There is no code, no pseudocode for the agent loop, and no controller prompt in the appendix [...] Can the code, the agent loop and the controller prompts be released, or summarised as pseudocode, so that the search procedure and the retained procedures can be inspected?

Yes. We give the complete workflow of the system as two pieces of pseudocode, in the notation of Section 3. Algorithm 1 (this part) is one round of research: how candidate models are proposed, implemented and tested. Algorithm 2 (Part 2) is Scientific Evolution: how procedures are proposed from the research records, and what condition a procedure must meet to be retained. The end of Part 2 also gives one procedure that was actually retained. Upon acceptance we will release the full code, the prompts of every agent and all procedures retained in the experiments.

Algorithm 1  One round of EvoSim (Section 3.2)
Input:  model M_t = (P_t, θ_t), measurements y_exp,
        research state R_t, scientist state S_t
Output: M_{t+1}, R_{t+1}, S_{t+1}

1: λ_t ← ŷ_t − y_exp
   # residual of the current model
2: ({h_{t,j}}, ρ_t) ← π_{S_t}(M_t, λ_t, R_t)                  # Eq. (2)
   # h_{t,j}: a model change and its expected effect on predictions
   # ρ_t: research plan (Focused Investigation or Parallel Comparison,
   #      conditions held fixed, evaluation procedure, compute budget)
3: for j = 1..m_t in parallel:
4:   (M'_{t,j}, τ_{t,j}) ← I(M_t, h_{t,j}, ρ_t; S_t)           # Eq. (2)
     # implement on a copy of M_t, run the solver, record trace τ_{t,j}
5:   E_{t,j} ← ScientificValidation(M'_{t,j})
     # (a) specification: are the intended changes active?
     # (b) numerical: solver status and required outputs
     # (c) experimental: task error metric against y_exp
     # (d) assessment: what the result supports about h_{t,j}
6:   a_{t,j} ← 1 if E_{t,j} meets the acceptance criteria in ρ_t, else 0
7: M_{t+1} ← Select_{ρ_t}(M_t, {(M'_{t,j}, E_{t,j}) : a_{t,j} = 1})  # Eq. (3)
   # M_{t+1} = M_t if no candidate is accepted
8: R_{t+1} ← R_t ∪ {(h_{t,j}, τ_{t,j}, E_{t,j})} for all j
   # rejected candidates are recorded too
9: S_{t+1} ← ScientificEvolution(S_t, R_{t+1})                # Algorithm 2
   # runs even when M_{t+1} = M_t

Algorithm 1 describes one round of research. A round does three things: it proposes how to change the model; it implements the changes and tests them; it records the results.

First, proposing changes (lines 1-2). The system compares the predictions of the current model with the measured data and finds the discrepancy. For this discrepancy it proposes several changes. Each change states two things: what to change in the model; and how the predictions should change afterwards.

Second, implementing and testing (lines 3-5). The changes are pursued at the same time. Each change is implemented on a copy of the current model and solved, which gives a candidate model. Every candidate model goes through four checks: whether the change is actually active in the solved model; whether the solver completed normally; what the error between predictions and measurements is; and whether the result supports or refutes the change.

Third, accepting and recording (lines 6-9). The system selects the new current model from the candidate models that pass the checks. If no candidate passes, the current model is unchanged. Whether or not a candidate passes, its change, its solving process and its check results are recorded, and Algorithm 2 learns from these records. A round that does not improve the model therefore still leaves experience.

AuthorsOpenReview ↗

[Part 2/7] Author Response

Weakness 1 & Question 1 (cont.): [...] so that the search procedure and the retained procedures can be inspected?

Algorithm 2 is Scientific Evolution. Its input is the research records left by Algorithm 1. Its output is the updated scientist state S, i.e. everything the system stores for later use.

Algorithm 2  Scientific Evolution (Section 3.3)
Input:  scientist state S_t, research state R_{t+1}
Output: S_{t+1}

 1: ΔS+ ← ∅, ΔS− ← ∅
    # (i) learn from every investigation, accepted or rejected
 2: for each investigation (h, τ, E) in R_{t+1}:
 3:   add to ΔS+ the findings and failure experience supported by E,
      each with the conditions under which it was observed
 4:   if the observed result departs from the expected effect of h:
 5:     identify the modeling step or research decision responsible
 6:     propose a correction c =
        (what to change, expected effect, conditions of use, checks)
 7:     record c in R_{t+1} as a proposal   # untested: not yet in S
    # (ii) test the proposals that were applied in this round
 8: for each proposal c applied in this round:
 9:   if the evidence supports the expected effect of c:
10:     add c to ΔS+ as a procedure with its conditions of use
11:     add to ΔS− the earlier entry that c revises
        # the failure experience behind it is kept
12:   else:
13:     add to ΔS+ the failure of c with its conditions
14:     revise c using the new feedback; it stays a proposal
15: S_{t+1} ← (S_t \ ΔS−) ∪ ΔS+                               # Eq. (4)
16: return S_{t+1}       # used by π_{S_{t+1}} in the next round, Eq. (5)

A correction c can revise, all under the rule of lines 8-10:
  knowledge: physics-based findings, failure experience
  skills:    procedures, tools, checks
  multi-agent orchestration:
             research strategies (how many candidate branches run in
             parallel, over how many rounds, how the final answer is
             chosen) and the instructions given to each agent

Algorithm 2 describes how the system learns from these records. It does two things: it proposes corrections from the records; it tests the corrections and decides whether to retain them.

First, proposing corrections (lines 2-7). The system reads every record of the round, whether or not the candidate model was accepted. It first records the findings and the failure experience of the investigation. If the actual result of a change departs from what was expected, it identifies which modeling step went wrong and proposes a correction. A correction states four things: what to change; what effect is expected; under what conditions to use it; and which checks confirm it. At this point the correction is only a proposal and does not enter S.

Second, testing corrections (lines 8-14). After a proposal has been applied in later research, the system checks whether its expected effect appeared. If it did, the proposal becomes a procedure and enters S (lines 9-10). If it did not, the failure is recorded as experience, and the proposal is revised and waits to be tested again (lines 13-14). This is the retention rule of Section 3.3: a practice is retained because its expected effect was observed in use, not because it worked once.

What is updated. The end of Algorithm 2 lists the three kinds of content that a correction can change, which the abstract calls knowledge, skills and multi-agent orchestration. Knowledge means physics findings and failure experience. Skills means procedures, tools and checks. Multi-agent orchestration means research strategies and the instructions given to each agent. A research strategy specifies how an investigation is organized: how many candidate branches run in parallel, over how many rounds, and how the final answer is chosen. A "candidate branch" (Section 3.2) is each of the investigations run in parallel in line 3 of Algorithm 1. All three kinds must pass the same retention rule.

A retained procedure. Below is one procedure retained in the experiment of Part 3, listed by the four items of line 6.

  1. What to change: before fitting, check whether the raw reading is already the quantity to be found. If it is, fit it directly; if not, convert it first.
  2. Expected effect: the submitted law no longer carries the extra factor introduced by the conversion.
  3. Conditions of use: tasks in which the instrument returns a numerical reading. The conversion must not be applied on the basis of the name of the law's category alone.
  4. Checks: verify the fitted law on new experimental points.

The appendix of the revision will include these two algorithms, the prompts of every agent and all procedures retained in the experiments.

AuthorsOpenReview ↗

[Part 3/7] Author Response

Weakness 2 & Question 2: The experiments are too narrow. [...] Nothing in the paper shows whether the procedure transfers to a different physical system, so the claim of general physics modelling rests on one domain. Can the evaluation include physical systems beyond lithium-ion cells, so that the transfer of the procedure, rather than its fit to one model family, can be assessed?

Yes. We added an experiment on NewtonBench [3]; it answers both this comment and the next one.

Task. NewtonBench is a benchmark for physical law discovery. Each task hides a physical law. The agent collects noisy data through simulated experiments and submits the law it finds. A submission counts as correct only if it is mathematically equivalent to the hidden law. The tasks cover 12 physics domains outside batteries: gravitation, Coulomb force, magnetic force, Fourier heat conduction, Snell refraction, radioactive decay, underdamped oscillation, Malus's law, speed of sound, Hooke's law, the Bose-Einstein distribution and heat transfer. The law of each domain has 3 versions. Each version has 3 observation modes: the output of the law can be observed directly; the law is hidden inside a simple physical system; the law is hidden inside a complex physical system. In the last two modes only the output of the system is observed, so the system must be modeled before the law can be found. We use all tasks of the hard level, 12 × 3 × 3 = 108 in total.

Protocol. The experiment is a cross-validation over law versions: procedures are learned from the tasks of two versions and tested on the third. There are three steps. First, an agent with no experience attempts each of the 108 tasks once and leaves research records. While working on a task the agent does not know whether it is right; after the task ends, each record is labeled correct or wrong. Second, EvoSim builds procedures from these labeled records following Algorithm 2: it proposes procedures, tries them on tasks, and retains only the procedures whose expected effect appears. Third, testing: EvoSim attempts each of the 108 tasks once with the procedures. For a task of one version, the procedures it uses come only from the records and labels of tasks of the other two versions. The 3 versions of a law are 3 different laws, so the answer to a tested law has not been given to EvoSim before testing. Halfway through testing it updates the procedures once more following Algorithm 2. The records read by this update carry no correct-or-wrong label. The agent without experience and EvoSim use the same base model and the same per-task budget.

Table A. Number of tasks on which the correct law is found.

MethodDirect observation (36)Simple system (36)Complex system (36)Total (108)
Agent without experience2213944
EvoSim32332388

Result. On 12 physics domains outside batteries, EvoSim finds 88 of the 108 laws; the agent without experience finds 44.

What transfers is the method itself. What runs on these domains is the two algorithms of Parts 1 and 2: competing hypotheses are investigated in parallel; candidate models are accepted or rejected on evidence; procedures are proposed from the research records; only procedures whose expected effect appears are retained. We added no hand-written knowledge specific to these domains; all procedures were obtained by EvoSim from the research records. These tasks do not use COMSOL either, so the result does not depend on the commercial solver used in the paper.

Procedures transfer across domains. An example: EvoSim had stored a procedure that converts the instrument reading once before fitting. In one gravitation task the reading itself is the quantity to be found. All candidate branches still applied the conversion, and every submitted law carried the same extra factor. The update during testing read the research records of 6 other domains and added conditions of use to this procedure. The result is the procedure listed at the end of Part 2. After that, all 6 candidate branches found the correct law.

Reviewer_ZNMbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvoSim combines physics-model development with reusable procedure learning. It revises mechanisms and equations using simulation feedback, then retains validated procedures for later investigations. Battery-modeling experiments report improved predictions and fewer failures across 45 paired runs.

Critical strengths
  1. The system performs actual solver-based modeling with implementation, numerical, and physical checks. Separating model acceptance from procedure retention is a sensible design.
  2. The repeated experiments provide meaningful evidence: model/physics failures decrease from 33 to 21 across 45 pairs, remaining significant after multiple-testing correction (Appendix D.4).
Critical weaknesses
  1. Only two of the 16 held-out voltage results are reported. The headline 7.62 mV result therefore does not establish performance across the stated cross-validation protocol (Appendix C.4.5).
  2. The comparison supports actionable procedural memory, but does not establish an advantage over generic experience distillation. The principal three-condition comparison also uses only one run per condition.
  3. Claims of outperforming expert models rely on literature values with different calibration procedures and, often, different conditions. Table 10 does not provide a controlled comparison supporting this claim.
Questions for authors
  1. What are the results for all 16 held-out voltage traces, including their aggregate error?
  2. How does Scientific Evolution compare with a generic experience-distillation baseline under matched source histories and budgets across repeated runs?
  3. How does EvoSim compare with expert models under identical data splits and calibration budgets?
4 comments
AuthorsOpenReview ↗

[Part 1/4] Author Response

Thank you for your careful review and specific comments. We first summarize three results.

  1. All 16 held-out voltage traces are now reported. EvoSim has the lowest error of the 9 models on every trace, with a mean of 17.49 mV (Table A).
  2. All models in Table A use the same data split and calibration budget, so it is also the controlled comparison with expert models.
  3. We added two generic experience-distillation baselines, ExpeL and AWM, on the 108 tasks of the physical-law-discovery benchmark NewtonBench: EvoSim finds 88 laws, ExpeL 62 and AWM 60 (Table B). The comparison of Section 4.3 now includes both baselines, with 3 runs per condition (Table C).

Our detailed responses follow.


Weakness 1 & Question 1: Only two of the 16 held-out voltage results are reported. The headline 7.62 mV result therefore does not establish performance across the stated cross-validation protocol (Appendix C.4.5). What are the results for all 16 held-out voltage traces, including their aggregate error?

The task is to predict the terminal voltage of a battery under dynamic current. DST and US06 are two standard dynamic profiles in which the current changes rapidly. A model is scored by the root-mean-square error (RMSE) between predicted and measured voltage, in millivolts (mV); lower is better. "Calibration" means determining the model parameters from measured data. A "held-out trace" is a trace that is not used for calibration and is used only for testing.

Table A reports all 16 held-out voltage traces. The evaluation is the cross-validation of Appendix C.4.5: the data are grouped by 8 temperatures; each time the 2 traces of one temperature are held out for testing and the model is calibrated on the 14 traces of the other 7 temperatures. All models use the same A123 data, the same split and the same calibration budget.

Table A. Held-out voltage RMSE (mV). Each row is a temperature (°C) and profile.

TraceEvoSimECM-LUTCPG-SPMT (orig.)2RC-KiBaMESPCPG-SPMT (refit)NDCTNetPIROL2RC ECM
−10 DST33.4457.9855.6950.6161.2657.9478.4477.2668.03
−10 US0632.1454.7259.5248.2672.3864.4161.0381.1662.15
0 DST24.0431.3844.2562.1442.1544.1041.8866.3465.26
0 US0619.8926.2441.9850.0438.8342.5134.6959.1553.22
10 DST14.4723.4130.8220.7028.6130.6528.3343.7051.66
10 US0618.4520.0234.6024.1232.5133.1835.6245.0460.71
20 DST20.5626.9336.7034.4036.6937.2045.0542.8653.66
20 US068.9714.0414.5620.4913.3016.3125.6418.5330.21
25 DST17.8529.3932.0731.4132.5634.0542.8033.9042.35
25 US0611.3421.5520.6820.4821.1521.7736.7122.5235.47
30 DST7.6218.8121.5621.7624.1925.1930.6522.6026.74
30 US0613.1719.2719.8018.6621.9321.8433.3520.4428.17
40 DST13.4318.4829.2628.8334.6135.0730.0236.0124.99
40 US0618.1819.7421.6422.7826.3524.6339.8722.0523.37
50 DST12.2826.9626.1035.0733.4135.4825.4038.1626.54
50 US0613.9620.7424.6228.5832.0234.7625.0537.4321.83
Mean17.4926.8532.1232.4034.5034.9438.4141.7042.15

The lowest error on each trace is in bold. The baselines are described and cited in Part 2.

The mean RMSE over the 16 traces is 17.49 mV and the median is 16.16 mV. EvoSim has the lowest error on every trace, so the conclusion does not depend on which traces are shown. Low temperature is the hardest condition for all models. At −10 °C the error of EvoSim is 33.44 and 32.14 mV; that of the model with the second-lowest error is 50.61 and 48.26 mV. The revision will report the 16-trace mean in the abstract and include Table A in the main text.

AuthorsOpenReview ↗

[Part 2/4] Author Response

Weakness 3 & Question 3: Claims of outperforming expert models rely on literature values with different calibration procedures and, often, different conditions. Table 10 does not provide a controlled comparison supporting this claim. How does EvoSim compare with expert models under identical data splits and calibration budgets?

Table A (Part 1) is this controlled comparison: every model uses the same data split and the same calibration budget. Three conclusions follow.

  1. EvoSim has the lowest error on all 16 traces. The expert models compared include CPG-SPMT, PIROL, NDCTNet, ESP and 2RC-KiBaM, published between 2024 and 2026.
  2. The mean of EvoSim is 17.49 mV, 35% below the second-best model (26.85 mV) and 50% below CPG-SPMT recalibrated on every fold (34.94 mV).
  3. The model built by EvoSim is physics-based. Its error is below that of the data-driven NDCTNet on every trace.

The baselines. They cover three classes of models: equivalent-circuit models (ECM-LUT, 2RC ECM, 2RC-KiBaM), reduced electrochemical models (CPG-SPMT, ESP, PIROL) and a data-driven model (NDCTNet). Their references are CPG-SPMT [1], 2RC-KiBaM [2], ESP [3], NDCTNet [4], PIROL [5] and 2RC ECM [6]. CPG-SPMT (orig.) uses the authors' parameters; (refit) is recalibrated on every fold. ECM-LUT is a second-order RC equivalent circuit whose parameters are tabulated over state of charge.

Every baseline is reproduced strictly following the model and the calibration method of its paper. For example, CPG-SPMT with the authors' parameters gives 32.07 mV on 25 °C DST; the published value is 32.10 mV. Cui et al. [7] is not included, because it forecasts later voltage from voltage that has already been measured, which is a different task. The implementation details of every baseline will be listed in the appendix. The revision will replace Table 10 with Table A as the basis for the comparison with expert models.


References

[1] CPG-SPMT: Control-oriented parameter-grouped single particle model with thermal effects for Lithium-Ion batteries, Computer Physics Communications 2026

[2] Kinetic model for improved dynamic current response in lithium-ion battery electrical circuit models, Monatshefte für Chemie 2025

[3] An extended single-particle model of lithium-ion batteries based on simplified solid-liquid diffusion process, iScience 2024

[4] Remaining discharge energy prediction for lithium-ion batteries over broad current ranges: A machine learning approach, Applied Energy 2024

[5] A physics-informed reduced-order lumped aging framework for lithium-ion batteries under real-world EV driving cycles, IEEE Open Journal of Vehicular Technology 2026

[6] Low-cost experimental validation of lithium-ion battery models and SOC estimators under dynamic current profiles, Clean Technologies 2026

[7] Long-sequence voltage series forecasting for internal short circuit early detection of lithium-ion batteries, Patterns 2023

AuthorsOpenReview ↗

[Part 3/4] Author Response

Weakness 2 & Question 2: The comparison supports actionable procedural memory, but does not establish an advantage over generic experience distillation. The principal three-condition comparison also uses only one run per condition. How does Scientific Evolution compare with a generic experience-distillation baseline under matched source histories and budgets across repeated runs?

This comment has two parts: the comparison with generic experience distillation, and the single run of the three-condition comparison. We ran a new experiment for each. The first is answered below and at the start of Part 4, the second in Part 4. Below, EvoSim denotes the full system, which includes Scientific Evolution.

(1) Comparison with generic experience-distillation methods

Baselines. ExpeL [8] distills text insights from successful and failed tasks. Agent Workflow Memory (AWM) [9] induces reusable workflows from successful tasks.

Task. We use NewtonBench [10], a benchmark for physical law discovery. Each task hides a physical law. The law has been deliberately altered, for example in an exponent, so it cannot be answered by recalling a textbook. The agent collects data through simulated experiments and submits the law it finds. The tasks cover 12 physics domains outside batteries, so this experiment tests the learning method itself rather than battery knowledge. We use all 108 tasks of the hard level. They fall into three classes of 36: the output of the law can be observed directly; the law is hidden inside a simple physical system; the law is hidden inside a complex physical system. In the last two classes only the output of the system is observed, so the system must be modeled before the law can be found.

Protocol. All methods use the same source records and the same budget. There are three steps. First, an agent with no experience attempts each of the 108 tasks once and leaves research records. While working on a task the agent does not know whether it is right; after the task ends, each record is labeled correct or wrong. Second, each method builds its own memory from these labeled records. Each law has 3 altered versions, which are 3 different laws. For a task of one version, a method uses only the records and labels of tasks of the other two versions. The answer to a tested law has therefore not been given to any method before testing. Third, testing: each method attempts each of the 108 tasks once with its memory. During testing EvoSim also reads the research records of the tasks it has already finished and updates what it has stored once. These records carry no correct-or-wrong label. This update is part of the method. All methods use the same base model and the same per-task budget.

Table B. Number of tasks on which the correct law is found.

MethodDirect observation (36)Simple system (36)Complex system (36)Total (108)
No experience2213944
ExpeL3419962
AWM31191060
EvoSim32332388

Result. EvoSim finds 88 laws, twice as many as without experience (44), 26 more than ExpeL and 28 more than AWM.

Where the gap appears. When the output of the law can be observed directly, the three methods are close (EvoSim 32, ExpeL 34, AWM 31). When the system must be modeled first, the gap opens: 33 against 19 and 19 on simple systems, and 23 against 9 and 10 on complex systems. On complex systems the two baselines are at the same level as without experience (9).

Why the gap. The learning signal of ExpeL and AWM is sparse: when a task ends there is one outcome, success or failure, and the method learns from that one outcome. Scientific modeling is a long and hard process, and looking only at the final outcome discards the many valuable signals in its intermediate steps. EvoSim differs in three ways. First, its learning signal is dense: within a task it tests every candidate model, and every test leaves evidence, including how far predictions are from measurements and whether the result satisfies physical constraints. EvoSim learns from this evidence rather than only from the final outcome. Second, EvoSim calls each stored practice a procedure. Before a procedure is stored it is tried on other tasks, and it is retained only if its expected effect appears. Third, EvoSim updates not only the stored procedures but also the research strategy, i.e. how an investigation is organized, for example how many candidate branches (Section 3.2) investigate in parallel. Each candidate branch is one independent investigation of the same task.

AuthorsOpenReview ↗

[Part 4/4] Author Response

Weakness 2 & Question 2 (cont.): [...] How does Scientific Evolution compare with a generic experience-distillation baseline under matched source histories and budgets across repeated runs?

An example. Malus's law has 3 complex-system tasks in the experiment of Part 3. Before the update during testing, EvoSim had 18 candidate branches on these 3 tasks. All of them fitted the data down to the noise level, but 17 of them had fitted the wrong physical quantity. Records made during testing carry no correct-or-wrong label. By final fit error all 17 count as successes, so a method that learns only from success or failure receives no signal here that anything needs correcting. The update of EvoSim reads the test evidence of every candidate model in the research records. From the records of another physics domain it obtained a procedure: first write the equation from the law to be found to the instrument output, then fit. With this procedure, 16 of the 18 candidate branches found the correct law and all 3 tasks were solved.

(2) The three-condition comparison of Section 4.3, with 3 runs per condition

This task starts from a faulty voltage model: its predicted voltage differs from the measurements by about 89 mV. The system has to find the fault and repair the model. The three conditions in the submission are Baseline (no experience), Naive Memory (a chronological summary of the records) and Scientific Evolution. We added ExpeL and AWM and ran every condition 3 times (gpt-5.6-sol, 30 °C DST, at most 3 solves per run).

Table C. Lowest valid voltage RMSE obtained in each run (mV).

ConditionRun 1Run 2Run 3Runs repaired
Baseline89.4889.29394.880/3
Naive Memoryno valid solution89.2089.380/3
AWM89.2983.6489.570/3
ExpeLno valid solution89.3821.561/3
Scientific Evolutionno valid solution21.5621.562/3

Scientific Evolution repairs the model in 2 of the 3 runs, reducing the error to 21.56 mV, consistent with the 22.02 mV in the submission. ExpeL repairs it once. AWM, Naive Memory and Baseline never repair it, and their error stays at about 89 mV. The ordering of the methods is the same as in Table B. Tables B and C and the discussion of ExpeL and AWM will be added to the revision.


Thank you for recognizing in your review that the system genuinely builds models with a solver and applies three kinds of checks, and that model acceptance is separated from procedure retention. The three responses above are each supported by a controlled experiment (Tables A, B and C). If anything remains unclear, please tell us and we will follow up within the discussion period. If these responses resolve your concerns, we would be grateful if you would consider raising your rating.


References

[8] ExpeL: LLM Agents Are Experiential Learners, AAAI 2024

[9] Agent Workflow Memory, ICML 2025

[10] NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents, ICLR 2026

Comments on the paper

3 comments

For all revierers: as the discussion period is approaching its end, please join the discussion and put acknowledge or feedback on authors' rebuttal, thank you!

AuthorsOpenReview ↗

[Part 1/2] Final Author Summary for the Area Chair

We thank the Area Chair and reviewers for their careful evaluation and constructive suggestions. As the author response period draws to a close, we offer this two-part summary to help the Area Chair consider the additional evidence alongside the original reviews.

  • Part 1 highlights EvoSim's scientific significance and methodological contributions.
  • Part 2 summarizes how our responses and additional experiments address each of the reviewers' weaknesses and questions.

1. Scientific Significance and Main Contributions

EvoSim revises physics-based models from experimental evidence while learning how to conduct subsequent investigations. Physical modeling requires choosing which processes enter the equations and how they jointly determine measured behavior. A discrepancy may arise from parameters, implementation, or physical assumptions. EvoSim investigates these alternatives by implementing candidate revisions, checking their execution and physical consistency, and comparing predictions with measurements. The results determine which revisions to retain and what to investigate next.

Two industrial battery tasks show what these decisions achieve. The fast-charging task predicts the battery's state of charge (charge percentage) at the experimentally defined onset of lithium deposition on the negative electrode. EvoSim compared two assumptions about the fraction of deposited lithium that can no longer participate in charging and discharging: a constant fraction, or one that varies with charging rate. Guided by prediction errors across rates, it selected the latter, reducing held-out mean absolute error from 3.79 to 1.79 percentage points of state of charge.

The second task predicts battery voltage under changing vehicle loads. To address errors after current changes, EvoSim added equations for ion movement and charge accumulation at electrode surfaces. With the same data splits and budget for parameter fitting, the final model outperformed eight baseline configurations on all 16 held-out traces. The mean per-trace root mean square error (RMSE) was 17.49 mV, versus 26.85 mV for the best baseline, a reduction of about 35%.

The investigations also provide evidence for improving research methods. Scientific Evolution has three distinguishing features.

First, it learns from every candidate-model test: prediction discrepancies, checks that proposed changes are active in the executed model, and checks of physical constraints. Rejected candidates also inform proposed improvements.

Second, each proposed procedure specifies an action, an expected effect, conditions of use and checks. It is tried on tasks not used to propose it and retained only when its expected effect is observed. Procedure retention is separate from model acceptance, so a rejected model can still yield a useful research method.

Third, the same rule governs updates to knowledge, modeling procedures and checks, agent instructions and research strategies. These include how many candidates to investigate in parallel, how many rounds to run, and how to select the final result. Updated methods guide later investigations, whose results inform further revisions. This is EvoSim's form of recursive self-improvement (RSI) in scientific modeling.

Controlled experiments test whether the learned methods improve subsequent research. Across 45 matched pairs of runs on 11 battery-modeling tasks, using retained procedures reduced the number of runs with modeling or physics failures from 33 to 21.

The added NewtonBench study covers 108 tasks in 12 non-battery physics domains, requiring inference of deliberately altered laws through simulated experiments. EvoSim, ExpeL and AWM use the same source records, base model and per-task execution budget. With all memories fixed during testing, they solve 86, 62 and 60 tasks, respectively. On the 36 complex-system tasks that require modeling the system before identifying the target law, the counts are 21, 9 and 10. Continued updates raise EvoSim's total to 88; the examples summarized in Part 2 show how these updates change subsequent modeling decisions.

The battery experiments demonstrate autonomous revisions of physics-based models against real measurements. The cross-domain comparisons test whether research experience improves later tasks. Together, they support a method for scientific agents to improve both the models they build and the way they conduct subsequent research.

AuthorsOpenReview ↗

[Part 2/2] Final Author Summary for the Area Chair

2. Responses to the Reviewers’ Weaknesses and Questions

We summarize how the added evidence addresses each original weakness (W) and question (Q), in review order.

ZNMb (R1)

W1/Q1 — Results for all 16 held-out voltage traces.

We report all 16 traces under the original leave-one-temperature-out protocol. Mean RMSE is 17.49 mV and the median is 16.16 mV. EvoSim has the lowest error on every trace, supporting performance across the full evaluation.

Reply Part 1/4.

W2/Q2 — Matched experience-learning baselines and repeated runs.

ExpeL, AWM and EvoSim use the same source records, base model and per-task budget. With all memories fixed during testing, EvoSim solves 86/108 NewtonBench tasks, versus 62 for ExpeL and 60 for AWM. In three voltage-repair runs per condition, EvoSim succeeds in 2/3 and ExpeL in 1/3; AWM, Naive Memory and the no-experience baseline each succeed in 0/3.

Reply Parts 3-4/4; frozen comparison: XMnw Part 7/7.

W3/Q3 — Controlled comparison with expert models.

We compare eight baselines under the same data splits and budget for parameter fitting. EvoSim has the lowest error on all 16 traces: mean RMSE 17.49 mV, versus 26.85 mV for the best baseline. This provides a direct comparison under matched conditions.

Reply Part 2/4.

XMnw (R2)

W1/Q1 — Inspectable research process and retention rule.

Two pseudocode algorithms specify how models are proposed, implemented, tested and selected, and how procedures are proposed, tried and retained. An actual retained procedure states its action, expected effect, conditions of use and checks. Both model selection and procedure retention can therefore be inspected.

Reply Parts 1-2/7.

W2/Q2 — Physical systems beyond batteries.

We evaluate the same research and learning method on 108 hard NewtonBench tasks in 12 non-battery physics domains, without COMSOL. With the same base model and per-task budget, EvoSim with continued updates solves 88 tasks, versus 44 without experience. Cross-domain procedure transfer is illustrated under R3 W2.

Reply Parts 3-4/7.

W3/Q3 — Altered laws to test inference beyond recall.

The benchmark changes exponents or constants in physical laws. For each test version, we build the initial procedure library using only records from the other two versions. A submitted law is correct only if it is mathematically equivalent to the hidden law. This tests inference of altered laws, complementing the battery evidence on model-structure revision.

Reply Part 4/7.

W4/Q4 — Retention, parallel investigation and the procedure library.

Relative to the frozen-library result of 86, removing the retention test gives 81, using one branch gives 73, and removing the library gives 44. Each ablation changes one component. The single branch receives the entire budget previously shared by the branches, testing parallel investigation at the same task budget.

Reply Part 5/7.

W5/Q5 — Differences from Voyager, ExpeL and AWM.

EvoSim learns from each candidate's prediction errors, implementation checks and physical checks. A procedure is retained only if its own expected effect appears on tasks not used to propose it. Updates include knowledge, skills, agent instructions and research strategies: branch counts, rounds and final-result selection. R1 W2 reports the matched ExpeL/AWM comparison.

Reply Parts 6-7/7.

6xEv (R3)

W1/Q1 — Methodological novelty and stronger, fair baselines.

A procedure must state an action, conditions of use and an expected effect, which is tested before retention. Removing this test reduces the result from 86 to 81. R2 W5 explains the three methodological differences; R1 W2 gives the matched strong-baseline and repeated-run comparisons.

Reply Parts 1-3/5.

W2/Q2 — Continued updates and evidence of model selection.

With the same initial library and per-task budget, continued updates give 88 correct tasks versus 86 with the library fixed. Among 21 tasks using modified procedures, 20 are correct with updates versus 17 without. Updates use test-time research records without correctness labels.

In three complex-system Malus tasks, the instrument measures the combined output of several stages, while the target is a single-stage law. A procedure learned from Snell-refraction records first models how the stages combine to produce the output, then fits the single-stage law using this model. Applying it changes the result from 0/3 to 3/3 correct. The update thus changes the model connecting the target law to observations.

Reply Parts 4-5/5.


We thank the Area Chair and reviewers again for their careful evaluation and constructive suggestions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.