acceptodds
Under review as a conference paper at ICLR 2027

GEMOT: Towards Mechanistic World Models For Biology

12 views

Abstract

Scientific discovery seeks mechanisms that explain observations and predict beyond the measurements that produced them. Whereas large language models (LLMs) encode knowledge implicitly, mechanistic world models organise it as a parsimonious set of explicit, modular, reusable mechanisms whose predictions can be scored against data. In biology, where measurements are sparse and noisy, mechanistic world modelling must discover latent states and governing equations jointly, yet the prior knowledge that could constrain this search is largely unstructured. We introduce gemot, an agentic framework for mechanistic world modelling, and evaluate it on 18 published biological problems spanning molecular biology, epidemiology, and immune-cell differentiation, with up to 1,755 training measurements, 65 observables, 170 experimental conditions, and 3 data modalities per problem. gemot is auditable by construction: a semantic layer records each hypothesis, and a Model Context Protocol layer separates hypothesis formulation from numerical evaluation, so hypothesis scoring cannot be fabricated. On every problem, autonomously constructed models match or exceed the reference models in fit and parsimony, and generalise where held-outs exist. We demonstrate that gemot can formulate novel, biologically plausible mechanistic hypotheses. This work will enable decoding of interventional biological data into competing mechanistic hypotheses and design experiments that distinguish between them, and may serve as a blueprint for mechanistic world modelling in other domains.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Mechanistic World Models, Systems Biology, AI for Science, Scientific Discovery, LLM Agents, Epidemiology, Cell Signalling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XwmfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents GEMOT, an LLM-based framework for automatically developing and refining mechanistic models of biological systems through iterative hypothesis generation and numerical evaluation. The approach is evaluated on 18 published biological modelling tasks, demonstrating competitive model quality and promising predictive capabilities on selected held-out experiments. The manuscript is well written, clearly motivated, and supported by extensive experimentation and detailed methodological documentation.

The main limitations concern the consistency of generalization across benchmarks, the difficulty of establishing genuine mechanistic novelty, and the limited evidence of improvements over standalone frontier LLM agents. Overall, this is a valuable and timely contribution with promising results, although some claims would benefit from stronger experimental support.

Critical strengths
  1. The paper is clearly written, well motivated, and supported by extensive methodological details, making the proposed framework easy to follow and its experimental findings easier to assess.

  2. The authors evaluate the framework across 18 diverse, real-world biological modelling problems, comparing against established reference models. The inclusion of repeated runs, held-out experimental validation, and ablation studies across different LLMs and agent configurations provides substantial evidence of its practical capabilities.

  3. The proposed architecture separates LLM-driven hypothesis generation from numerical evaluation, enabling transparent and traceable model construction. This is particularly valuable for scientific applications, where numerical reliability and interpretability are essential, and provides a promising foundation for automated mechanistic discovery.

Critical weaknesses
  1. Although the paper includes held-out experimental evaluations and discusses several biologically plausible hypotheses, the proposed mechanisms are not independently validated through new biological experiments. Moreover, the qualitative analysis focuses on selected representative examples, making it difficult to assess how consistently the framework generates biologically meaningful hypotheses. A more systematic evaluation of mechanistic plausibility would strengthen the scientific claims.

  2. The the proposed framework provides valuable architectural improvements, however the ablation experiments show that a standalone GPT-6-Astra agent achieves comparable, and in some cases better, predictive performance. This raises questions about the extent to which the observed improvements can be attributed to the proposed framework rather than the capabilities of the underlying LLM. A more detailed analysis of the individual architectural components would help clarify their contributions.

  3. While the authors compare GLM-5.3-Flash and GPT-6-Astra in their ablation experiments, the main evaluation across 18 biological problems relies exclusively on GLM-5.3-Flash. Evaluating the framework more extensively with different LLM families would help establish whether the observed performance generalizes across models and better isolate the contributions of the proposed architecture from those of the underlying LLM.

Questions for authors
  1. Could the authors elaborate on the motivation for selecting GLM-5.3-Flash for the main evaluation across 18 biological problems? Was this decision primarily driven by computational cost, or were other factors considered? Given the stronger performance observed with GPT-6-Astra in some experiments, it would be interesting to understand how the choice of LLM affects the framework's overall capabilities.

  2. Could the authors further clarify which components of GEMOT provide the most significant improvements over a general-purpose LLM agent? In particular, are the main benefits related to model quality, reliability, computational efficiency, or reproducibility?

3 comments

W1: Plausibility shown only for selected examples. We now compare every model with curated knowledge (OmniPath, Reactome) on 13 problems, three of them new, not selected examples (Sec. 4.3, Apps. D.4 and F). Where the agents’ models depart from it, an LLM-assisted audit suggests gaps in the curated networks, justifiable coarse-graining (e.g. EGFR acting on Akt without PI3K), genuine misses and limitations of the analysis; how to judge such departures remains an open problem (Sec. 4.3). The structural comparison (Fig. 2, Table 11) already covers all runs. We agree that experimental validation is the definitive test.

W2: A bare GPT-6-Astra agent performs comparably. In response to the concerns about organisation (Reviewer WwtP, W1), we removed the Fujita bare-agent configurations and the analysis of hardcoded literals to shorten the text and improve the narrative flow. The revision instead tests the harness on the six largest problems with published references (Sec. 4.6), where it matters:

  • (i) With GLM, barebones agents passed 2/30 runs against 29/30 with gemot, and the harness’s feedback made search more efficient (Fig. 5).

  • (ii) gemot lets an open-weight model reach 98% of the best frontier improvement at under 3 USD per model, compared with over 6 USD (Fig. 3e).

  • (iii) gemot computes scores outside the LLM, and the agent cannot alter them, so it cannot fabricate results; this makes results more trustworthy and reduces the need for human audit (App. B.4).

W3: Single LLM family in the main evaluation. GLM-5.3-Flash was the first open-weight model that consistently solved all problems, though no longer the only one. We evaluated Astra on Fujita only because of cost, which grows on larger problems where fits can exceed its 1 h cache timeout. The revision adds Claude Fable 5.2 on Fujita; both frontier LLMs were faster than GLM but not more predictive (6/10 against 10/15 models beat the reference; Sec. 4.4). We added DeepSeek v4.1 Flash (open-weight), GPT-6 luna and Claude Haiku 5.5 on the six largest problems with published references (Fig. 5): DeepSeek and Haiku passed 29/30 runs, as GLM did, and luna 14/30, most of its failed runs exhausting the token budget.

Q1: Why GLM-5.3-Flash? It was the first LLM that consistently solved the original 18 problems, and at under 3 USD per model (Fig. 3e) it made 110 runs across 22 problems affordable. See also W3.

Q2: Which components matter most? The improvement likely results from many small, compounding gains rather than any single component, and the computational cost of each run makes a comprehensive deconstruction of performance difficult. Based on the ablations (Figs. 3 and 5):

  • Performance: barebones agents passed 2/30 runs against 29/30 with gemot on the six largest problems with published references (Sec. 4.6).

  • Cost: open-weight models become competitive (98% of the best frontier improvement at less than half the cost).

  • Auditability: gemot versions every model and computes scores outside the LLM.

  • Search signals: withholding fix pointers or progress signals left success unchanged but made the search less efficient (Fig. 5): without fix pointers, agents had to derive revisions themselves; without progress signals, they lost track of their best model and branched off stale versions.

Removing the curator had little effect (held-out wins 4/5 vs. 3/5), consistent with the LLM’s implicit knowledge being sufficient for Fujita. Biological context mattered for prediction on some systems but not others (Sec. 4.5).

I appreciate the work done by the authors to address most of my presented concerns. However, I decide to maintain my score.

We thank the reviewer for their engagement and are pleased that our response addressed most of their concerns.

Reviewer_WwtPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper builds on the recently introduced theoretical/methodological framework of Mechanistic world models from Posner, Lei, and Schölkopf [1], and proposes to operationalize it for automated building of mechanistic models from data, in biology specifically, through an agentic system. On a range of 18 published biological models, the proposed strategy matches or exceeds the reference (fully human-designed) model through the formulation of novel biologically relevant hypotheses and mechanisms.

[1] From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery, arXiv, 2026.

Critical strengths
  • I enjoyed reading the introduction section. I believe it clearly outlined the task of constructing mechanistic models in biology, as well as its many challenges. As someone with expertise in the "old school design" of mechanistic models for systems biology, I found that quite convincing.
  • The results are encouraging: on a wide array of biological/data scarcity/heterogeneity settings, the models found by the agentic framework match or exceed the performance of the reference model, which was designed by hand by a human modeller. Not only that, but the analysis carried out by the authors found that these are not the result of pure memorization. Instead, the agentic framework can sometimes propose novel and relevant mechanisms/hypotheses.
Critical weaknesses

-Past the introduction section, the paper is badly organized and simply not in an advanced enough state for publication, and I am not sure the necessary changes are "small enough" that a rebuttal phase alone will fix that. Several sections currently in the appendix would need to be in the main text, e.g., Appendix A.3, so that we clearly understand the type of mathematical object the approach produces here. I understand that these agentic systems involve many moving parts, making it cumbersome to provide a concise description that will fit a conference paper, but not even having a single equation in the main text seems a bit extreme. Even if the mathematical framework was defined in [1], having a refresher in the main text would help.

  • The "Limits of published benchmarks" paragraph in the discussion is a bit concerning and shows that it is still difficult to provide an unbiased assessment of these methods when using public benchmarks, but at least this is acknowledged by the authors.
  • There is no comparison to existing approaches, except in Figure 4 against a distilled neural ODE symbolic regressor.
  • The appendix is more than 200 pages long, and still contains sentences that make it look like an internal technical report (e.g., beginning of appendix B, appendix D.5, and more). I see in the reproducibility statement that the appendix is intended as some kind of crude proxy for a code repository, but still, it really feels like this was thrown there with little polishing.

[1] From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery, arXiv, 2026.

Questions for authors
  • Is there something in the setting that prevents comparison with other approaches, e.g., LLM-based symbolic regression approaches like LLM-ODE? I must say that I am not very familiar with the recent literature on agentic systems for inferring ODE models/mechanistic models.

I do not have any other questions. My concerns relate to the paper's organization and less to the method itself.

3 comments

W1: Organisation; no equations in the main text. We agree. The PEtab formulation (App. A.3, Eqs. 1–3) is now in Sec. 2, which relates it to the mechanistic world model notation (App. B.3). We also introduced an evaluation design section (Sec. 3) that guides the reader through the rest of the manuscript; other content is largely unchanged except for updates from reruns.

W2: Limits of published benchmarks. We added two problems without a published model (Fontaine et al. 2026; Lun et al. 2019), on which the agents’ models fit the data closely (Sec. 4.2). We see no performance difference between problems with and without a published reference model (Sec. 5).

W3 / Q1: No comparison to existing approaches. LLM-based symbolic regression methods (LLM-SR, LLM-ODE) assume direct observations of the state variables. Our problems require inferring latent states and an observation mapping to up to 65 observables, 3 modalities and 9,169 conditions. We have added a table classifying related methods by whether they handle latent states, multiple modalities and multiple conditions (Table 6). To our knowledge, there are no other methods that can solve the problem we set out (Eq. 1) at the scale of the reference models, up to the 1,228-state Fröhlich model, so we compare against barebones agents and distilled neural ODEs (Sec. 3).

W4: Appendix. We did not condense the appendix but removed internal provenance detail (revision stamps, the App. B introduction, superseded September results) and shortened the audit-ledger notes to one statement per record. It had already undergone extensive human review for the first submission, and we are happy to take concrete suggestions on how to better present such a large body of work. Its notes that not all results come from the same code version are an honest account that the code would have made evident anyway; long run times and costly evaluations prevented rerunning everything after each change. For the camera-ready, we will rerun everything from one final version.

I acknowledge the author's rebuttal, the paper is now in a better shape, but could still benefit from many more rounds of polishing to make it easier to read. I will maintain my score. I am not fully convinced that a conference proceeding (10 pages), with an appendix of more than 50 pages of text, and ~200 pages of additional appendices, is the optimal venue for this work.

We thank the reviewer for acknowledging the improvements. We are happy to continue polishing for the camera-ready version and would welcome pointers to specific passages the reviewer found hard to read or not self-contained. On the venue: the main text is self-contained and supports its claims within the page limit, as ICLR’s format intends. The appendices give background for readers who want more depth, and the records needed to reproduce every result; neither is required to follow the paper.

Reviewer_tDSZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Gemot is an agentic model that uses LLMs to turn biological knowledge and experimental data into the variable, mechanism and structure discovery. The major technical contributions lie in the architectural design and adoption of judgement, evaluation, semantic, and insulation layers. Gemot was tested on 18 problems from the PEtab Benchmark Models, spanning molecular biology, interacting cell populations, and epidemiology. It was found to propose novel mechanistic hypotheses, some of which are biologically plausible. It was also found to accompany prediction accuracy with mechanistic underdetermination when data is limited.

Critical strengths
  • The study aims at the joint learning of variable, structure, and binding for mechanistic hypotheses; and is making a sound step toward a mechanistic world model where knowledge and measurements update to triangulate mechanistic hypotheses.
Critical weaknesses
  • The evaluations are relatively weak. Specifically, the problems are of limited systems biology scales, the assessment of hypotheses is indirect and insufficient, and the metric of NLL alone is of limited information.

  • Even though the world model as future milestones is out of the scope, simulating and evaluating how mechanistic hypotheses evolve as measurements accumulate or update would much strengthen the study, provided more rigorous and systematic evaluations.

Questions for authors

Could the authors please provide stronger evidences with more rigorous and systematic evaluations, as detailed in the weaknesses above?

3 comments

W1a: Limited scale. We added the CCLE dataset of Fröhlich et al. (2018): 143 cell lines, 8 drugs, 9,169 measurements, which is ~5× the largest current problem (1,755 training measurements). All five runs beat the published model (ΔBIC ≈ −37,200) with ~160 instead of 4,231 parameters, at under $1 each; three other LLMs did so in 14 of 15 runs (Secs. 4.2 and 4.6).

W1b: Hypothesis assessment is indirect. The submission already compares the structure of every candidate with its reference across all 90 defined comparisons; 76/90 differ in connectivity or latent depth (Fig. 2; Table 11). We replaced the literature-based analysis of six selected systems (Table 8) with a comparison of every model with curated knowledge (OmniPath, Reactome) on 13 of the 22 problems, one of them (Lun) without a published model; the agents’ models agree with it about as well as the published ones (Fig. 2, Sec. 4.3). An LLM-assisted literature audit of each link that departs from curated knowledge and each curated connection a model misses complements it as a sanity check (App. F).

W1c: NLL alone is limited. We report training BIC, AIC and NLL for all 110 runs (Fig. 2): 89/100 runs with a published reference match or improve on it by BIC, and at least one run has lower BIC on 19 of 20 problems (all but Blasi). We also report held-out prediction on eight benchmarks (Figs. 3–4) and structural differences (Fig. 2). We now summarise these in the main text.

W2: Evolution of hypotheses with data. We agree this is an interesting direction, but it does not bear on the claims of the paper, so we leave it to future work.

I thank the authors for the responses.

  • The CCLE experiment is a helpful addition.
  • Assessment of mechanistic hypotheses remain a concern. In particular, can the predicted hypotheses be evaluated definitely and directly.
    • The level of agreement/disagreement with known mechanisms is hard to interpret at this moment. The claim was that even though the learned hypotheses rarely agree with known mechanisms completely, they could be alternative mechanisms. Is it a wrong mechanism or an alternative mechanism? Is the amount of measurement data adequate or inadequate to determine the known mechanisms? If under-determined what qualifies an alternative mechanism besides "no-data-conflict" and how difficult is it to find one of such qualified hypotheses? These and similar questions await to be answered to interpret the evaluation results.
    • The connectivity evaluation and literature audit both focus on individual connections/links that collectively form mechanistic hypotheses, that is, components of a hypothesis rather than a complete hypothesis itself.
  • NLL evaluates likelihood and AIC/BIC reflects parsimony. Meanwhile NLL as an average over all cases can hide distinctive failure modes; and NLL also does not separate accuracy and calibration. Could more metrics be included for what NLL misses?

We thank the reviewer for specifying the remaining concerns.

Wrong vs. alternative mechanism. The only direct and definitive test of a mechanism is experimental data. We deliver a retrospective evaluation and show its problem with published data: most published held-out data do not discriminate between competing hypotheses (Sec. 4.5). Prospective validation is beyond the scope of this manuscript and not pertinent to the claims we make.

Links vs. complete hypotheses. Complete models are evaluated as wholes, by fit and parsimony (Sec. 4.2) and by held-out prediction (Secs. 4.4, 4.5). The link-level comparison (Sec. 4.3) adds whether their parts are biologically plausible.

Beyond NLL. After every fit, the agent receives per-observable diagnostics (App. B.7): sign and runs tests for bias and temporal structure, replicate lack-of-fit, plausibility of estimated noise scales and, where noise is supplied, χ² adequacy. As these data cannot separate technical noise from biological variability, we applied the same diagnostics to the published models. After Holm correction, residual structure is flagged in 17% of agent models and 17% of published models, and the χ² test never rejects an agent's model where it accepts the published one. We will report these diagnostics per model in the camera-ready version.

Comments on the paper

1 comment

We thank the reviewers. We have uploaded a revised PDF with the following changes:

  1. The problem formulation (App. A.3) is now in the main text (Sec. 2).

  2. Evaluation on four new problems: the CCLE dataset of Fröhlich et al. (2018), Dessauges et al. (2022), and two problems without a published mechanistic model (Fontaine et al. 2026; Lun et al. 2019).

  3. A rerun of all 22 problems, five runs each, and of the Fujita comparison with Claude Fable 5.2 added; updated Figures 2 and 3.

  4. An upgrade of the benchmark to PEtab v2, which enables problems such as Dessauges and reduces the protocol information agents must specify.

  5. Removal of potential information leaks from the prompts; all conclusions of the first submission still hold.

  6. Four more LLMs and further ablations on the six largest problems with published references.

  7. A capability comparison with related methods.

  8. A comparison of every model with curated knowledge (OmniPath, Reactome) on 13 problems.

Several requested analyses were already in the submission but were not prominent enough. We have moved them into the main text and reference them below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.