acceptodds
Under review as a conference paper at ICLR 2027

ALDER: Discovering the Laws of a World by Acting in It

Abstract

Reliable world models should not only predict future states but express how actions change the world in an explicit, transparent and testable form, such as equations. Yet methods that rely on a fixed set of trajectories cannot distinguish equally good competing hypotheses, while searches over a fixed set of predefined candidates cannot discover equations outside the initial hypothesis space. We introduce ALDER (Action-guided Law Discovery, Evaluation, and Revision), a method that actively proposes novel experiments to test and revise models. Specifically, ALDER proposes parametric equations; a numerical optimizer fits their coefficients; an independent verifier tests these candidates on held-out data. To distinguish between competing valid hypotheses, a cost- and safety-aware selector queries interventions, in the form of novel experiments. The resulting counterexamples update the evidence ledger and guide the next structural revision, while incompatible laws are discarded. Across an in-house benchmark, ODE equation discovery tasks, and robotic experiments, ALDER discovers laws beyond its initial formula set, repairs failed model proposals, distinguishes fixed candidate models with fewer interactions, and improves out-of-distribution prediction. Furthermore, given a current state and a target, ALDER selects control actions by solving the inverse problem defined by its validated world model. Together, these results show that explicit equation-based world models can be tested and revised through interaction, then naturally used to guide goal-directed control.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Scientific law discovery, Symbolic regression, Active experimentation, World models, Hypothesis revision, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Wp1uOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper uses an LLM to propose equations (world models), fits their coefficients numerically and has an independent verifier admit or reject each candidate. When several hypotheses remain, it actively chooses new experiments to tell them apart. Experiments cover a synthetic benchmark (NovelLaw), simulated robot tasks (ManiSkill, RoboCasa), and ODEBench/ODEBase.

Critical strengths
  • The pipeline is cleanly designed, with the proposer, fitter, and verifier separated. The agent is sandboxed, and refusals count as failures. The appendix also honestly reports negative results, such as no universal ARS advantage on RoboCasa. The refusal analysis (RQ4) is useful.
Critical weaknesses
  • "Discovering laws beyond the initial set" is an overclaim. In NovelLaw, the held-out targets are just new combinations of components already seen during development. For RoboCasa, the grammar in the code allows only bias and velocity plus at most 2 of 11 fixed terms (67 structures), which include exactly the MuJoCo terms. The non-LLM enumeration in the code even does better than the LLM agent.
  • Active experimentation does not show a clear benefit, and some evaluations look leaky. On RQ1, ALDER ties random querying (85.0% vs. 85.0%, p=0.203) while using more queries. The RQ2 gain uses a pool deliberately seeded with irrelevant points. In the code, admission requires shift NMSE ≤ 0.02, and the reported shift NMSE seems to come from the same split, which conflicts with Sec. 3.3. For ODE tasks, the agent receives exact derivatives (not stated in A.6.1), while the baselines are copied from other papers rather than rerun. All robot results are in simulation.
  • The novelty relative to LLM-based equation discovery and classical model discrimination is unclear. The propose-fit-verify loop is very close to LLM-SR (Shojaee et al., 2025) and SR-Scientist, both cited but not compared. Choosing experiments by disagreement among candidate models is classical model discrimination / query-by-committee (QBC appears only as a baseline copied from prior work on ODEBase). The baselines needed to isolate ALDER's contribution are missing: LLM-SR and SR-Scientist on NovelLaw and the robot tasks with the same data and grammar, PySR/SINDy run on ALDER's actively collected data, an expected-information-gain query selector, and grammar enumeration on RQ3.
Questions for authors
  1. How do PySR or other LLM-based equation discovery methods perform on NovelLaw with the same data and grammar? How does plain enumeration do on RQ3?
  2. Is the reported NovelLaw shift NMSE computed on a split that is not used for admission?
  3. Did the ODE baselines receive exact derivatives? What happens with noisy, numerically differentiated data?
  4. How do LLM-SR and SR-Scientist perform on NovelLaw and RoboCasa under the same data and interaction budget?
5 comments

Thank you for your thoughtful review and for highlighting the separation of proposal, fitting, and verification, the sandbox, and the refusal analysis. ALDER integrates revisable proposals, evidence acquisition, protected admission, and downstream use within a supplied grammar. NovelLaw tests new compositions of available components. RoboCasa uses physical features in simulation. We build on disagreement-based model discrimination.

1. NovelLaw baselines and RoboCasa enumeration

On all 60 NovelLaw cases, baselines received the public fitting and development observations acquired by ALDER. Hyperparameter and program selection used public data. We accessed test points only after fixing models.

MethodIndependent Shift recovery
Frozen ALDER reference50/60
SINDy29/60
PySR (real-power operators)28/60
LLM-SR (12-call adapter)21/60
SR-Scientist (12-turn adapter)36/60

Recovery requires test NMSE ≤ 0.02 and counts failures. We report the declared configurations and retain both PySR operator settings and shorter LLM runs. SINDy uses a fixed library, PySR searches over operators, ALDER uses a typed AST, and the LLM adapters use their own program interfaces. We document each search budget. The frozen ALDER reference retains its proposal and admission history.

In RoboCasa, enumeration of the 67 allowed structures with ALDER’s acquired data and public-validation selection admitted 38/40 Drawer and 14/40 Door, versus 40/40 each for frozen ALDER. Other selection variants match ALDER’s Drawer coverage. With robot evidence fixed, admissions in Drawer/Door were 35/40 and 31/40 with the original grammar, 28/40 and 20/40 without the linear position term, 18/40 and 13/40 without both friction-sign terms, and 37/40 and 30/40 with ten added terms. These ablations measure sensitivity to the supplied grammar.

2. Independent NovelLaw Shift evaluation and acquisition

We used the original Shift split for admission and will label its scores as admission-set diagnostics. We evaluated the frozen final models on 4,096 new points per case in each domain, reserved for final testing. On the same 60 target equations, ALDER recovered 50/60 on independent Shift (median NMSE 0.00630) and 51/60 on the broader functional domain. We retained all cases and failures.

In 60 NovelLaw cases (three paired runs each), ARS and random shared initial proposals, fitted coefficients, and budgets. Functional recovery was 97/180 vs 95/180. In the observation-use comparison, mean extra points were 3.33 vs 3.73 per run (10.7% fewer for ARS).

With online stopping, ARS vs random identified 195/200 vs 189/200 on uniform pools and 194/200 vs 186/200 on Latin hypercube pools, averaging 1.085 vs 1.735 and 1.065 vs 1.830 queries, respectively. Monte Carlo expected information gain identified 195/200 on both pools, averaging 1.110 and 1.135 queries.

3. ODE derivatives and observation sensitivity

The original ALDER runs used exact derivatives. The literature baseline scores alone leave derivative access unclear. We reran full ALDER and baselines on 12 prespecified systems stratified by dataset and dimension. With exact derivatives, shared trajectory evaluation, and blinded symbolic scoring, functional recovery was 11/12 ALDER, 6/12 LLM-SR, and 8/12 SR-Scientist. We report all 14 methods or variants, budgets, and failures. Passive methods received ALDER’s final public data. Active methods shared a 20-point start and could query up to two more 20-point trajectories.

A separate fixed-data study used the same systems and trajectory evaluator.

ObservationsStructural proposerSINDyPySR
Exact derivatives6/121/120/12
Clean numerical differentiation3/120/120/12
Numerical differentiation, 0.1% state noise1/120/120/12
Numerical differentiation, 1% state noise0/120/120/12

Differentiation used fixed numpy.gradient. All models were frozen before testing. Recovery requires trajectory NMSE ≤ 10⁻⁶ in reconstruction, generalization to unseen initial conditions, and contiguous future extrapolation.

4. LLM-SR and SR-Scientist with shared observations

The comparison requested here was completed on NovelLaw only. On the independent Shift test, moving from the same passive observations to ALDER’s final public observations raised LLM-SR recovery from 8/60 to 21/60 and SR-Scientist from 21/60 to 36/60. Their acquired data came from ALDER-selected interventions.

The 12-call LLM-SR adapter retains the pinned official island sampler and buffer. The 12-turn SR-Scientist adapter retains the official prompts, tools, and history interface. The adapters share backend and reasoning settings. We document grammar, program size, and fitting adaptations, along with the ALDER reference’s historical settings.

AuthorsOpenReview ↗

Equation Search versus Active Observation Acquisition

Thank you for encouraging direct comparisons with LLM-SR and SR-Scientist under shared observations and interaction budgets. Your question helps us clarify an important distinction between iterative equation search and active observation acquisition.

The standard workflows of LLM-SR and SR-Scientist operate on supplied observations. LLM-SR improves equations through candidate search and numerical fitting, while SR-Scientist uses data-analysis tools, equation evaluation, and iterative feedback. Neither standard workflow includes an acquisition mechanism that autonomously selects and executes environmental interventions to obtain new observations. SR-Scientist’s autonomous tool use concerns analysis and equation refinement on available data, rather than active acquisition of environmental evidence.

ALDER connects equation discovery with active observation acquisition: disagreement among candidate models can guide the next intervention, and the resulting observations support further equation revision.

Accordingly, in our reported NovelLaw shared-observation comparison, we provide both methods with the public observations acquired by ALDER, allowing their respective equation-search procedures to operate on identical evidence through the described adapters. This evaluates equation discovery from shared observations; it does not attribute ALDER’s intervention choices to either baseline or compare independent acquisition policies.

AuthorsOpenReview ↗

Additional RoboCasa Results: Comparison with LLM-SR and SR-Scientist under Matched Observations

We thank the reviewer for this constructive suggestion, which motivated us to strengthen the direct comparison with LLM-SR and SR-Scientist. Building on our earlier NovelLaw experiments, we have now completed the comparison across all 80 RoboCasa environments, comprising 40 Drawer and 40 Door environments.

Both baselines receive exactly the same public observations acquired by ALDER, while retaining their respective search and numerical fitting procedures. ALDER’s candidate equations, fitted coefficients, and protected validation feedback are not provided to the baselines. All three methods use a common LLM backbone, with at most 12 LLM calls per environment across the entire discovery process. All new baseline models are frozen before evaluation on the same independent test trajectories. Recovery requires force-prediction NMSE ≤ 0.03.

MethodDrawerDoorOverall recovery
ALDER38/4040/4078/80 (97.5%)
LLM-SR37/4029/4066/80 (82.5%)
SR-Scientist39/4026/4065/80 (81.25%)

With identical observations and the same call cap, ALDER achieves 97.5% overall recovery, exceeding LLM-SR and SR-Scientist by 15 and 16.25 percentage points, respectively. The improvement is particularly clear on Door, where ALDER recovers all 40 environments, compared with 29 for LLM-SR and 26 for SR-Scientist. SR-Scientist also performs strongly on Drawer, recovering 39/40 environments.

We appreciate the reviewer’s recommendation: this additional comparison strengthens the empirical evidence for ALDER’s ability to recover predictive equations from limited robotic observations and clarifies its performance relative to existing LLM-based discovery methods.

Reviewer_DoUvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to learn explicit, equation-based world models through interaction. ALDER combines four components. An agent proposes and revises equation structures within a supplied grammar. Numerical fitting estimates their coefficients. Disagreement-based experiment selection guides data collection. Independent validation can reject unsupported models.

The paper presents an integrated framework for grammar-constrained symbolic model discovery, active experiment selection, validation with abstention, and prediction and action selection through numerical inversion. The evaluation covers compositional equation discovery, robotic interaction tasks, public ODE benchmarks, and fixed-candidate model identification. It also examines RoboCasa prediction and control under changed dynamics. Across these settings, the results suggest that ALDER can improve model discovery, prediction, and control performance, while also indicating the conditions and limitations under which these gains hold.

Critical strengths
  • The RoboCasa prediction results are strong. ALDER substantially improves shifted prediction over the matched MLP and approaches the known-parameter reference. The models share acquired evidence and use the same inverse-control solver budget. This provides useful evidence that compact equation-based models can offer an effective inductive bias for changed joint dynamics.

  • The fixed-candidate acquisition study is informative. Holding the candidate models and query budgets fixed gives a relatively clean comparison of acquisition strategies. ARS reduces mean penalized identification cost from 3.53 for random selection to 1.16. It also improves over coverage and parameter uncertainty, though the latter gain is smaller.

  • The system connects discovery, validation, and downstream use coherently. Separating structural proposals from fitting and admission is sensible. So is refusing models that fail validation. The PullCubeTool example illustrates how contradictory evidence can lead to a useful structural revision. I also appreciate that failed runs remain in the reported results.

Critical weaknesses
  • The novelty appears mainly at the integration level. LLM-SR already combines equation proposals with numerical fitting, and SR-Scientist studies agentic equation refinement. The connection to LLM-ACES is particularly close: disagreement-selected trajectories feed back into symbolic search. These works are cited and acknowledged by the authors, yet I feel the framing sounds more ambitious than what is actually contributed. ALDER adds direct structural proposals, robotic interventions, and protected admission, but the rest of the components (which are the more central components in my opinion) and motivation is established prior art.

  • This can be a legitimate systems contribution. However, its value then depends heavily on showing what the integration adds. The experiments do not fully isolate those additions. In particular, the benefits of the cost and safety terms remain unclear, and the admission mechanism is not compared with simpler validation checks.

  • The acquisition advantage is less clear during structural discovery. On NovelLaw, recovery rises from 43.3% for one-shot proposals to 75% for revision-only, then to 85% for both random-query and ALDER. The shifted-error difference between the last two is unresolved (p=0.203). They also use independent proposal calls. The harder robot tracks show the same aggregate admission count for random-query and ALDER, without a resolved shifted-error difference.

  • These results support revision and collecting additional observations. They provide less evidence for the specific acquisition rule during structural discovery. RQ2 isolates that rule, but with supplied candidates. Its identification times are also retrospective: every run executes all eight queries. The improvement over parameter uncertainty is modest, with mean cost 1.16 versus 1.35 and ties in 161 of 182 cases.

  • The ODE comparisons leave the source of the large gains unclear. The reported symbolic-accuracy improvements are substantial. However, the reference methods were not rerun, the backbones and budgets differ, and ALDER uses a separate implementation of the symbolic evaluator. Using the same judge model does not ensure that the evaluation pipelines are equivalent.

  • The results support ALDER’s ability to recover these systems. They are less conclusive about its advantage over existing methods. The very small prediction errors do not resolve that comparison.

  • The robotic advantage is not separated from the supplied physical representation. RoboCasa uses low-dimensional joint dynamics, simulator state and force observations, and a fixed controller. The grammar supplies physically relevant features, while the neural baseline is a small MLP. Fitting or sparse selection over the same features would help establish what the agent’s structural search adds.

  • The passive-law comparison changes both the evidence and the opportunity for revision. It therefore does not separate their effects. Both tasks also use a median of 0.5 new queries per world, leaving the role of interaction worth clarifying.

  • I find the prediction improvement more consequential than the control-success gain. MLP success is already 96.9% and 99.5%. Practical scope also depends on the observation interface: no model passes admission with the tested motion-based force estimate. The authors report this limitation openly, but it matters to the application value.

  • The verification result is useful, but its distinct contribution is unclear. RQ4 admits all complete models and rejects all incomplete ones. This demonstrates useful refusal behavior under the tested conditions. It does not show repair of a fundamentally missing operator or state variable, since those experiments fix the available representation.

  • It is also unclear how much of the rejection result follows from an ordinary held-out error threshold. Residual localization is a separate diagnostic and fails to localize the missing smooth term reliably. Finally, the authors acknowledge possible overfitting through repeated validation use, but its effect on admission reliability remains unexamined.

Questions for authors

Questions relating to my assessment:

  • How does ALDER compare with fitting or sparse selection over the same RoboCasa physical features, using the same acquired evidence? This would help establish how much of the prediction advantage comes from structural discovery rather than the supplied representation.

  • Can the advantage over random acquisition during structural discovery be established with repeated proposal runs under the same protocol and budgets? The current comparisons support revision and additional observations, but leave the specific benefit of disagreement-based acquisition unresolved. Consistent gains here would strengthen the case for the proposed integration.

  • Can the large ODE symbolic-accuracy gains be confirmed using one evaluation pipeline for ALDER and the reference methods’ expressions? Please also clarify the differences in discovery budgets. This affects how much weight I can place on these comparisons.

Clarification questions:

  • In RQ4, which admission checks reject the incomplete models? Is rejection already explained by the held-out equation-error threshold, or do the other checks catch additional failures?

  • How are the cost and safety terms instantiated in the reported acquisition experiments? Do they affect the selected queries, or are their effects largely inactive in these settings?

1 comment

Thank you for highlighting RoboCasa’s prediction under shift and the fixed-candidate study. ALDER integrates proposal, fitting, querying, admission, and downstream use within a supplied grammar and interface. We will clarify ties to LLM-SR, SR-Scientist, and LLM-ACES.

1. RoboCasa with matched features and observations

On all 80 worlds, ridge and sparse regression use ALDER’s 13 physical features, acquired observations, admission threshold, and inverse-control solver budget.

MethodDrawer admissionDoor admission
Ridge45.0%2.5%
Sparse regression75.0%7.5%
ALDER (frozen reference)100.0%100.0%

ALDER’s reference models reflect iterative proposals and admission feedback. We will report coverage, shifted errors on admitted subsets, and control gains in percentage points.

2. Active querying

On 60 NovelLaw cases (three pairs each), ARS and random shared initial proposals, fits, and budgets. Functional recovery was 97/180 vs 95/180. In the observation-use comparison, mean extra points per run were 3.33 vs 3.73 (10.7% fewer for ARS).

On 20 Drawer pairs sharing an initially rejected proposal, ARS and random produced 20/20 vs 18/20 admissions with correct term support and averaged 1.55 vs 1.90 new queries. Initial admission in 20/40 worlds explains the 0.5-query median.

Our online fixed-candidate trials stopped using only the current posterior. In the 200-case mixed pool, ARS vs random achieved 194 vs 180 identifications, 1.085 vs 3.080 actual queries on average, and 1.35 vs 3.56 mean penalized costs. Uniform and Latin hypercube pools showed gains in the same direction.

3. Common ODE evaluation and budgets

We reran full ALDER and baselines on 12 prespecified systems stratified by dataset and dimension and saved their fitted equations.

MethodFunctional recoverySymbolic match
ALDER11/1211/12
LLM-only9/127/12
SR-Scientist8/124/12
LLM-SR6/125/12
Operon6/123/12
LLM-ACES (active)4/123/12
LLM-ODE2/121/12

We retained all variants. Recovery requires NMSE ≤ 10⁻⁶ in reconstruction, generalization to unseen initial conditions, and contiguous future extrapolation, with shared integration, normalization, and failure handling. Model-assisted symbolic scoring masks method identities and uses shared instructions and tolerances with fitted coefficients fixed.

We used exact derivatives. Passive methods received ALDER’s final public fit and development data. Active methods shared one 20-point trajectory and could query up to two more of equal length. LLM methods used gpt-5.6-sol (medium) and their own searches. ALDER used up to three outer proposal rounds. Per state coordinate, LLM-SR used up to 12 generation calls, SR-Scientist up to 12 rounds, and LLM-ODE 12 calls. We will label original literature scores separately.

4. RQ4 admission

Admission requires equation-only NMSE ≤ 0.02 on independent in-distribution validation data and mean residual-gate output < 0.10.

RuleComplete admittedIncomplete admitted
Equation error only20/200/60
Equation error + gate20/200/60

Both rules agreed on all 80 cases. The error threshold explains every rejection. Residual localization identified error regions for missing local terms or key states. For missing global smooth terms, shifted localization was near chance. Shifted AUROC is diagnostic only.

5. Cost and safety

RoboCasa scores queries by mean prediction disagreement divided by main plus prelude steps. Calls, tokens, and runtime are tracked separately. Fixed-candidate queries cost two ten-step trajectories each, so normalization preserves rankings.

We replayed ten ALDER decisions from selected worlds and reproduced their choices and scores. Without cost normalization at the same states, 6/6 Drawer choices changed and all four Door choices agreed. In deterministic loops on eight worlds per task, we toggled cost normalization.

TaskMean extra steps, with/without costReductionCorrect admissions in each arm
Drawer3.0 / 5.040.0%8/8
Door7.5 / 9.521.1%8/8

Query counts matched. Means include zero-query worlds and exclude shared initialization and reset costs.

RoboCasa uses action bounds, nominal simulation, and contact/workspace checks. In 40 fixed-candidate cases (four families, ten seeds each), we toggled screening on predicted state, speed, and acceleration. Both arms used the same numerical bounds, observation acceptance, and online stopping. Screening excluded 44/17,273 candidate evaluations and changed 9/45 top choices (20%) versus unfiltered scoring at the same states. The full-pool fallback never triggered. Each arm used 45 queries, identified 38/40 cases, and logged zero violations.

Reviewer_2eSUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ALDER learns equation-based world models by proposing candidate equation structures, fitting their coefficients, and checking their predictions against protected validation data. A language-model agent revises proposals using an evidence ledger. When more evidence is needed, an experiment selector chooses among feasible queries using disagreement between candidate models. Accepted equations can then be used to choose actions that reach a target state.

The authors evaluate equation discovery and prediction under distribution shift on NovelLaw and public differential-equation datasets. They also test prediction and control in simulated robot tasks. The experiments include comparisons with one-shot proposals, revision without queries, random query selection, symbolic regression, and neural prediction models.

Critical strengths
  1. The equation-based representation makes each proposed model executable and testable. The independent verifier can reject a candidate when the evidence does not support it. The robot experiments show lower prediction error under dynamics shifts than the matched neural model across the tested families. The accepted equations also support action selection.

  2. The evaluation distinguishes public evidence, query outcomes, protected validation data, and final test data. It includes rejected runs in the reported denominators and separates experiments on fixed candidate sets from the full discovery loop. These controls help distinguish equation revision from the contribution of query selection.

Critical weaknesses
  1. The full-system experiments leave the benefit of active query selection uncertain. On NovelLaw, ALDER and random-query discovery recover the same proportion of equations. The difference in shifted prediction error does not establish an advantage for ALDER (Appendix A.2, Tables 7–8). The corresponding robot comparison is also unresolved. The controlled study with fixed candidates supports the query-selection rule, but the full discovery loop needs a clearer account of when that rule contributes. In the RoboCasa structural-discovery experiment, the median number of new queries is only 0.5 (Appendix A.4.2).

The description of novel experiments also needs qualification. The evaluated queries are selected from predefined pools, with cached outcomes in the robot protocol. This is a useful experimental setting, but it leaves open how the method would construct an informative experiment beyond the supplied pool.

  1. The contribution of the benchmark-specific grammar is difficult to separate from the discovery procedure. Some supplied primitives encode substantial knowledge about contact or the dynamics being modeled. NovelLaw also checks that targets can be represented in the grammar. These choices make the problem well defined, but the experiments do not establish how sensitive discovery is to them. A comparison that removes an informative primitive or adds plausible distractors would help explain how much prior knowledge the method requires.

The scope of the results also depends on simulator access and sensing. The robot experiments use resets and directly measured forces, while the estimated-force condition admits no equations in Table 3B. Those results should inform the stated conditions under which ALDER can be used. The public ODE comparisons require separate care because the baseline numbers come from prior work and the symbolic-accuracy evaluation was reimplemented (Appendix A.6.1, Table 17).

Questions for authors
  1. When does active query selection improve the full discovery loop over random queries? Can you explain the similar recovery rates and uncertain prediction differences, including the low number of new queries used in RoboCasa?

  2. Who defines the grammar and query pool for each task? How much does performance depend on informative primitives already being present? What happens if a relevant primitive is removed or distractor operators are added?

  3. What counts as one interaction, and how were the query and proposal budgets chosen? How do the results change with those budgets? What additional feasibility or safety checks apply beyond membership in the predefined pool?

  4. How comparable are the public ODE results when baseline scores are taken from another study and symbolic accuracy is evaluated with a reimplementation? Which proposal model and prompts were used, and how do you assess possible familiarity with the published systems?

3 comments

Thank you for the thoughtful review and for highlighting ALDER’s executable equations, independent verification, and robotic predictions.

1. Active querying

Across 60 NovelLaw cases (three runs each), pairs shared initial proposals, fits, and budgets. Unlike the paper's semantic metric, recovery used independent Shift tests excluded from admission: 97/180 vs. 95/180 for ARS/random, with 3.33 vs 3.73 extra points (10.7% fewer for ARS).

In Drawer and Door, 20/40 cases each stopped at initial admission, explaining the median of 0.5 new queries. We compared 20 Drawer pairs whose shared initial proposals failed admission.

Drawer outcomeARS (ours)Random
Structure recovery (exact term support) ↑20/2018/20
Mean additional queries ↓1.551.90

ARS used 18.4% fewer queries in this subset. With fixed candidates and online posterior-based stopping, ARS averaged 1.085 vs 3.080 queries and identified the target in 194/200 vs 180/200 cases. Uniform and Latin hypercube pools also showed gains.

2. Grammar and query pool

Researchers set each task’s grammar and query pool in advance, shared across its worlds. The proposer receives only revealed outcomes, with targets withheld. Following your suggestion, we fixed evidence across 80 Drawer and Door worlds and varied the grammar.

GrammarAdmitted
Original66/80
Added ten distractors67/80
Removed linear-position term48/80
Removed both friction-sign terms31/80

Admissions held steady with distractors and fell without key terms. ALDER discovers and revises structures within the declared grammar. NovelLaw checks target representability.

3. Interaction budgets and feasibility

We preset matched limits. Shared initial evidence is counted separately.

ExperimentQuery unit and budget
NovelLaw24 points per batch, up to two batches and three proposal rounds
RoboCasaOne rollout per query, up to six new queries. Duration and prelude steps count.
Fixed candidatesTwo ten-step trajectories per query under one intervention, up to eight queries
New Drawer queriesAdmitted (ARS/random) ↑Admitted with correct support (ARS/random) ↑
237/40 vs 34/4031/40 vs 28/40
340/40 vs 38/4034/40 vs 31/40
540/40 vs 40/4034/40 vs 32/40

At a fixed proposal budget, ARS had more correct admissions at each logged checkpoint.

RoboCasa screens probes with a nominal mechanism and logs joint-position, handle-contact, and workspace violations. Fixed-candidate checks cover finiteness, acceleration, state bounds, and speed, with full-pool fallback if the predicted feasible set is empty. We will report fallbacks and violations. Queries select unused interventions from the pool. RoboCasa reveals cached outcomes after selection.

4. Direct ODE comparison

We share your concern about comparability. We reran full ALDER and baselines on 12 ODE systems selected in advance by dataset and dimension strata. Common trajectory and symbolic evaluators scored fresh equations, with method identities masked for symbolic scoring.

MethodFunctional recovery ↑Symbolic equivalence ↑
ALDER (full)11/1211/12
LLM-only9/127/12
SR-Scientist8/124/12
LLM-SR6/125/12
Operon6/123/12
LLM-ACES (active)4/123/12
SINDy (expanded)2/121/12
LLM-ODE2/121/12
SINDy (restricted)1/121/12
QBC†1/121/12
BO†1/121/12
PySR0/120/12
ODEFormer0/120/12
End2End0/120/12

† Our implementations of published algorithms.

Functional recovery requires NMSE ≤ 10⁻⁶ in reconstruction, generalization to unseen initial conditions, and contiguous future extrapolation. Symbolic scoring fixes fitted coefficients and uses a common tolerance. Models and configurations were frozen before final testing. Failures count in denominators.

We used exact derivatives. Passive baselines received ALDER’s final public fitting and development data. Active methods share a 20-point initial trajectory and can acquire up to two more 20-point trajectories. LLM methods use gpt-5.6-sol (medium) with their own search and fitting procedures. Total compute can differ. We will document calls, compute limits, operator libraries, and adaptations.

ALDER’s prompts request compact executable equations from anonymized states and derivatives and revisions based on public fit residuals and the evidence ledger. We withhold system names, reference equations, and protected test data. LLM baselines retain their prompt structures. We will provide prompts, call records, and per-system results. Anonymization reduces identity cues. Pretraining exposure remains possible.

ALDER led both measures on these systems. We will report these results separately from literature scores.

AuthorsOpenReview ↗

Additional results: direct comparison across all 122 ODE systems

Thank you again for encouraging a direct comparison under a common evaluation protocol. Following our initial response, we have completed the comparison on all 122 systems (63 ODEBench and 59 ODEBase), covering complete ALDER and 13 baseline configurations. The previously reported 12-system results are retained unchanged; the additional 110 systems extend the comparison to the full benchmark.

Evaluation protocol. All fitted models use the same trajectory integration and error computation. Functional recovery requires trajectory NMSE <= 1e-6 in all three settings: reconstruction, generalization to an unseen initial condition, and continuous future-time extrapolation. Symbolic recovery uses a common, method-blind, model-assisted equivalence assessment with fitted coefficients held fixed. All newly fitted models were frozen before final evaluation.

MethodFunctional recovery ↑Symbolic recovery ↑
ALDER (complete loop)106/122 (86.9%)106/122 (86.9%)
LLM-only85/122 (69.7%)71/122 (58.2%)
SR-Scientist83/122 (68.0%)49/122 (40.2%)
Operon74/122 (60.7%)28/122 (23.0%)
LLM-SR65/122 (53.3%)54/122 (44.3%)
LLM-ACES (active)42/122 (34.4%)30/122 (24.6%)
SINDy (expanded library)41/122 (33.6%)19/122 (15.6%)
LLM-ODE32/122 (26.2%)4/122 (3.3%)
BO (active, reconstructed)28/122 (23.0%)23/122 (18.9%)
QBC (active, reconstructed)21/122 (17.2%)19/122 (15.6%)
SINDy (restricted library)17/122 (13.9%)11/122 (9.0%)
PySR15/122 (12.3%)11/122 (9.0%)
ODEFormer1/122 (0.8%)0/122 (0.0%)
End2End1/122 (0.8%)0/122 (0.0%)

Observations and budgets. Passive baselines receive exactly the same final public fitting and development observations acquired by ALDER, allowing comparison of discovery performance on the same acquired evidence. Active methods start from the same 20-point trajectory and may acquire at most two additional 20-point trajectories, allowing comparison of complete discovery procedures under the same observation budget.

These experiments use an exact-derivative observation interface. All LLM components use gpt-5.6-sol with medium reasoning, while retaining method-specific prompts and search mechanisms. Computational budgets and expression spaces remain method-specific: these are budget-adapted implementations, rather than full-budget reproductions of the original papers. QBC and BO are reconstructed from published method descriptions. LLM-only iteratively proposes equations and fits coefficients on the shared data, without its own active acquisition loop.

Results. ALDER achieves the highest functional and symbolic recovery among the evaluated configurations. Compared with LLM-only, the strongest alternative on both metrics, ALDER recovers 21 additional systems functionally and 35 additional systems symbolically, improving recovery by 17.2 and 28.7 percentage points, respectively. The comparison also includes the requested LLM-based alternatives: LLM-SR, SR-Scientist, and LLM-ODE achieve 65, 83, and 32 functional recoveries, respectively, compared with ALDER's 106.

These completed results extend our initial subset evidence to the full 122-system benchmark and directly support ALDER's recovery performance under the stated observation conditions and common evaluation protocol.

Thank you again for the constructive discussion, which has made the paper clearer. So that all reviewers and the AC can see our latest clarifications, we summarize them briefly here.

  • Active selection in the full discovery loop reduces the interaction needed. In RoboCasa Drawer, ARS admits a model in all 40 worlds within three additional queries, while random querying needs five (Table 19). In the 20 worlds that need queries, ARS admits a model with the correct term support in 20/20 worlds against 18/20, with 18.4% fewer queries (Table 20). On NovelLaw, ARS reaches similar recovery with 10.7% fewer additional observations (Table 14).
  • Generic operators are sufficient on the public benchmarks. On all 122 ODE systems, ALDER uses only generic mathematical operators and recovers 106/122 systems (Table 1B). On the robot tasks, adding ten candidate terms leaves admissions essentially unchanged (66/80 vs. 67/80), and when a key term is missing the verifier withholds admission (Table 5).

We are very grateful for your suggestions. If these points address your remaining concerns, we would greatly appreciate it if you would consider raising your rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.