acceptodds
Withdrawn from ICLR 2027

An AI Agentic Framework for Automated Reconstruction and Analysis of Genome-Scale Metabolic Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Genome-scale metabolic modeling serves as a critical interface between genomic information and hypotheses regarding cellular functionality; however, its broader adoption is impeded by substantial programming demands and fragmented software ecosystems. In response, we present GemAgents, a language-agent framework engineered to automate the processes of metabolic modeling and analysis. This system allows biologists to express research objectives through natural language, thereby eliminating the necessity for manual coding of analytical workflows. GemAgents distinctly delineates the interpretation of language-based tasks and the orchestration of computational tools from the deterministic aspects of scientific computation, grounding model construction and evaluation in explicit input data, reaction evidence, and clearly defined modeling assumptions. The reconstruction pipeline includes sequence annotation, mapping of evidence to reactions, formulation of gene–protein–reaction associations, medium-constrained gap filling, and model exportation. An independent verification module evaluates mass and charge balance, identifies erroneous metabolite and energy generation, and employs a counterexample-guided repair strategy to suggest corrections that preserve specified metabolic functions. Furthermore, an integrated auto-analysis extension facilitates direct model interrogation, the design of non-native metabolic pathways, and the in silico integration and assessment of heterologous pathways. Engineering case studies utilizing publicly available genomes and reference models illustrate the robustness and versatility of these components. By combining natural language interaction with transparent, model-based computational processes, GemAgents offers a comprehensive solution that removes programming obstacles and effectively translates biological questions into experimentally testable computational hypotheses.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
LLM agents, automated scientific workflows, genome-scalemetabolic modeling, automated metabolic analysis, pathway design

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_8oYiOpenReview ↗
Rating
1
Confidence
3(+1)
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

GemAgents is an agentic framework that uses an LLM to interpret natural-language requests and orchestrate deterministic tools for genome-scale metabolic model reconstruction and analysis. It combines evidence-aware reconstruction, weighted gap filling, model auditing and counterexample-guided repair, and evaluates the resulting models against CarveMe, ModelSEED, and published references, with additional iML1515 and strain-design case studies.

Critical strengths

The paper presents a well-motivated separation between LLM-based orchestration and deterministic scientific computation. By constraining agent actions through explicit contracts, provenance tracking, and verification/repair modules, the framework reduces the risk that language-model errors directly propagate into scientific conclusions.

Critical weaknesses

The manuscript contains multiple unresolved internal inconsistencies that undermine confidence in the reported results. The most serious example is Figure 2: In result section, the author claims "GemAgents achieves a median score of 83.9, compared to 69.1 for CarveMe and 43.6 for ModelSEED Figure2". However, the actual Figure 2 is only a schematic of the GemAgents workflow and contains no benchmark data. Besides, in section 4.3, the author claims "For example, Flux Variability Analysis(FVA) ..." and Figure 4 is described as a succinate FSEOF analysis, while the plotted result is totally different. The manuscript also contains unresolved references such as “Appendix ??” and “(??)” in Algorithm 2.

Questions for authors

Could the authors clarify the actual content of Figure 2 and provide the missing benchmark results referenced in Section 4.1, including sample size and variability across reconstructed models?

Since most quantitative evaluations focus on the reconstructed metabolic models, could the authors clarify how the LLM-based agentic component itself was evaluated, and what additional capability it provides beyond the deterministic reconstruction and analysis pipeline?

2 comments

Thank you for your careful reading and for identifying these inconsistencies. We apologize for the confusion they caused. We have corrected the figure descriptions and unresolved internal references. The corresponding revisions are highlighted in blue in the revised manuscript.

  1. Figure 2 and Section 4.1. Figure 2 now presents the benchmark comparison: panel (a) shows the distributions of MEMOTE aggregate scores for CarveMe, ModelSEED, GemAgents, and the published-model group; panel (b) shows the corresponding predicted biomass objective outputs. The benchmark statement reports median scores of 83.9, 69.1, 43.6, and 75.8, respectively. The revised statement and caption are highlighted in blue on page 7, lines 352–354 and 368–374. The workflow schematic is Figure 1.

  2. Figure 4 and Section 4.3. We revised Section 4.3 and the Figure 4 caption to describe the lycopene FSEOF case, replacing the earlier description of succinate flux traces. Figure 4 displays published lycopene titers from Choi et al. (2010) for the dxs background and additional gene-amplification combinations. The revised text distinguishes this figure from the methodological description of FVA in Section 3.5. These revisions are highlighted in blue on page 9, lines 434–442, 457–462, and 464–470.

  3. Unresolved references in the Discussion. We replaced “Appendix ??” with explicit references to Section 4.3 and Figure 4 for the FSEOF workflow and Appendix C.2 for the production-envelope analysis. The corrected sentence is highlighted in blue on page 10, lines 511–512.

  4. Unresolved references in Algorithm 2. The two “(??)” references now resolve to “Equation (7)“ for parsimonious witness extraction and “Equation (9)“ for directional bound restrictions. The corrected algorithm lines are highlighted in blue on page 17, lines 888 and 892; the corresponding equations are highlighted on page 6, lines 271–275 and line 285.

We also checked the complete manuscript and recompiled it to confirm that no undefined citations or internal cross-references remain. We appreciate your feedback, which helped us improve the manuscript’s clarity and consistency.

Thank you for the corrections to Figures 2 and 4 and the unresolved references. These fix some presentation issues, but they do not address my main concerns about the missing benchmark details and the unclear evaluation of the LLM-based agentic component. I therefore maintain my original rating.

Reviewer_FjGSOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper presents GemAgents -- LLM agents that have access to tools that allow it to create metabolic pathways starting with some NL query.

The challenge is to produce metabolic models that have good biological fidelity. The authors argue that just putting an LLM in front of the tools naively isn't enough, and one needs GemAgents.

In GemAgents, LLMs orchestrate tool calls. The tools calls go through some kind of validator, and if they are valid, then the tool is run. There are also aspects of provenance tracking, counterexample-guided repair, and evidence-aware reconstruction.

For validation of the ideas, the paper show that (a) models generated by GemAgents have higher MEMOTE scores (Section 4.1), (b) reconstruction of a reference model, and (c) assessment of pathways.

Critical strengths
  • The idea of having a validator that validates any tool call generated by an LLM agent is interesting.
Critical weaknesses

The paper is presented at a very abstract level making it very difficult to understand the scientific contribution. From an AI/ML point of view, the finding here is that one should use LLMs to orchestrate domain-specific tools, and tool calls must be made only if a validator agrees. Apart from this, the rest of the technical details in the paper are about domain-specific tools -- which are poorly described.

The presentation follows a pattern of starting with some high-level concept with some name given to it, which is followed by a mathematical abstraction, but with no concrete instantiation. Since the main contribution here is a system architecture, it will be immensely helpful to provide one end-to-end worked example. This will concretize the mathematical abstraction and help readers appreciate the contributions. As one example, consider “contract-grounded orchestration”. This is supposed to distinguish GemAgents from ordinary LLM tool calling. But to fully understand this concept, the contract mechanism needs to be specified sufficiently for the reader to understand what is novel and reproduce it. Equation (2), together with statements that a validator V checks some k, doesn't by itself establish much of a technical contribution.

A challenge in Systems Biology is getting a measure of model validity. The evaluation here appears to lack an independent and scalable measure of the biological fidelity. This leads to what looks like contradiction. In Section 4.1 Line 265, for example, GemAgents is presented as outperforming manually curated models because it achieves higher MEMOTE scores, despite the paper's own observation that high MEMOTE scores do not imply biological fidelity. However, a little while later on Line 297, when evaluating biomass predictions, closeness to those same manually curated models is presented as evidence of biological accuracy. While these comparisons are not formally contradictory, they leave unclear what really indicates that a reconstructed model is biologically better.

Counterexample-guided repair is advertised initially, and then there is nothing said about it until Line 404 in the Discussion section.

Some more detailed comments- l135, goal g, scientific assets X, tools T, execution budget B. What are assets, and what tools are we talking about. Examples would help here. l135: provenance n. I happen to know stoihciometric matrices, bounds, and I can guess that a GPR rule would look like, but I can't predict what is provenance here. An example of GPR rule and provenance will help. l146: we already discussed validator checks and contracts above. l158: Example of evidence record? what is an "annotation", "source", "threshold" ? l190: "conjunction is compiled strictly" - what does compiling a Boolean operator 'strictly' mean? l250: cryptographic artifact checksum -- why do we need cyptographic checksum here? l361: "GemAgents pipeline maintains high resolution across alternate optima and endpoint flux variability" -- What does this mean?

Questions for authors

Please see the weakness section.

1 comment

Thank you for your precise feedback. We have revised the manuscript (highlighted in blue) to clarify tool interfaces, evaluation metrics, and counterexample-guided repair (CER). For reproducibility and further inspection, our code and data are available anonymously at: https://anonymous.4open.science/r/GemAgents-clean-1CF6/. Page/line numbers below refer to the revised draft.

  1. Concrete tool descriptions & execution mechanism. We clarified the distinct roles of domain-specific tools (e.g., NCBI/PGAP/HMM for annotation; CarveMe/Reconstructor for gap-filling; COBRApy/MEMOTE/FSEOF for analysis). Adapters return specific metadata: model hash, conditions, objective, solver status, and references (p. 5, lines 234–246; pp. 2–3, lines 106–116). We also explicitly defined the validator’s checks (e.g., tool/argument schema, asset hashes, execution budget), demonstrating how language requests strictly map to computational contracts (p. 4, lines 179–184).

  2. MEMOTE & biomass evaluations. MEMOTE provides standard GEM quality checks (Lieven et al., 2020). For annotation/SBO components, GemAgents (median 83.9) outperformed CarveMe (69.1) and ModelSEED (43.6), approaching the published-model group (75.8) (Section 4.1, p. 7, lines 342–354; p. 10, lines 498–499). The biomass comparison evaluates growth consistency against curated references. GemAgents (1.39) most closely matched the reference group (0.98) (Fig 2, pp. 7–8, lines 368–381). In the iML1515 case (Monk et al., 2017), GemAgents closely replicated reference aerobic/anaerobic growth (relative differences: 0.05% and 2.81%) with comparable MEMOTE scores (89.74% vs 90.60%) (Table 1, p. 8, lines 399–417). This confirms improved annotation and strong reference-behavior preservation.

  3. Counterexample-guided repair (CER). We added a concrete ATP-cycle example: a probe detects positive ATP, a witness finds supporting directions, and CER tests directional restrictions while validating the growth task (retained only if the task passes) (Section 3.4, p. 6, lines 294–301; p. 15, lines 794–800; p. 16, lines 844–851). Section 4.4 details two outcomes: successfully fixing a synthetic ATP-leak, and applying ten edits to an E. coli draft (leaving a residual cycle after budget exhaustion). Both cases highlight CER's practical utility and are retained in the evaluation (p. 9, lines 474–485; p. 10, lines 486–488).

  4. Terminology clarifications.

Assets/tools/budget: Defined as FASTA inputs, reaction catalogs, templates, plus limits on tool-calls, solver-steps, wall-time, and edits (p. 3, lines 143–148).

Evidence/provenance: Enumerated to include IDs, hashes, EC annotations, DB versions, and a ledger tracking accessions and transformation stages (p. 4, lines 201–212).

“Strict” GPR: Using (b1 AND b2) OR b3, we clarify that missing subunit evidence leaves the complex branch unresolved (p. 5, lines 227–233).

Reviewer_xWCAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GemAgents, a language-agent framework that aims to automate the processes of modeling and analysis of geome-scale metabolic models. Based on case studies utilizing published relevant work in genome-scale metabolic modeling, the authors demonstrate potentially efficacy and usefulness of the GemAgents framework and the versatility of the constituent components.

The manuscript shows that GemAgents can remove the various technical obstacles for genome-scale metabolic modeling and analysis for domain scientists by enabling natural language-based interactions to build computational solutions that can facilitate the prediction and interrogation of metabolic models.

Critical strengths
  1. The overall GemAgent framework is desinged in a reasonable manner, and an easy-to-use LLM-orchestrated framework that translates natural-language instructions to construct and analyze genome-scale metabolic models can be very useful for domain scientists who don't necessarily have the computational expertise to build an end-to-end modeling and analysis pipeline.

  2. Case studies and evaluations based on publicly available data show improved performance based on the MEMOTE scores when compared to existing approaches, such as CarveMe (published in 2018) and ModelSEED (published in 2021).

Critical weaknesses
  1. The evaluation process is not presented clearly and the performance evaluation results are not comprehensive enough. For example, there should be a detailed presentation and discussion regarding the translation of natural prompts to actual modeling and analysis outcomes. Is there sensitivity to the quality of the natural language instructions used? Are the outcomes and their quality robust enough to variations and potential ambiguity in the natural language prompts? What are the actual task success rates for different types of natural language-based modeling and/or analysis requests? To understand the value and efficacy of GemAgent framework, these aspects of the agentic framework should be clearly discussed.

  2. Current benchmarking results and metrics used do not convincinbly demonstrate whether GemAgent truly outperforms other alternatives considered in this work. For example, what aspects does the calculated "partial MEMOTE" score capture (e.g., consistency, biological accuracy, completeness, etc.)? How does GemAgent actually "generate high-quality metabolic models that closely match or exceedpublished standards" as claimed in Figure 2 caption?

  3. There are no guardrails discussed on how the authors ensure any potential data leakage issues in reproducing previously published results. As the evaluations are based on publicly available data and reproducing them, the benchmark results and case studies won't be very convincing unless these potential issues are clarified.

  4. In Sec. 4.2., the authors mention that "we conducted an in-depth case study using 4,298 MG1655 RefSeq proteins and a reference-assisted scaffold derived from the gold-standard iML1515 model." What was the "reference-assisted scaffold" and how was it used? Isn't it expected that "GemAgents achieve near-perfect architectural retention of the reference network" if an existing reference was used? As a result, it is unclear what the case study in Sec. 4.2 shows to demonstrate the efficacy of GemAgent.

  5. There should be concrete examples of natural prompts used to model and/or analyze genome-scale metabolic networks. Currently, the manuscript doesn't include any.

  6. How does (or can) GemAgent take and utilize input data and prior knowledge in the modeling and analysis process? What types of data and prior knowledge can be ingested/integrated into these processes?

Questions for authors

Please see the comments and questions in the critical weaknesses section above.

2 comments

Thank you for your careful review. We have clarified tool execution, metrics, reference usage, and added a concrete natural-language example. Revisions are highlighted in blue (page/line numbers refer to the revised 19-page manuscript). For reproducibility and further inspection, our code and data are available anonymously at: https://anonymous.4open.science/r/GemAgents-clean-1CF6/

1. Natural-language to modeling outcomes We detailed the contract-grounded execution (p. 4, lines 179–186; App B, p. 16, lines 812–822). Before dispatch, GemAgents validates requested tools, assets, source permissions, bounds, and execution budgets, explicitly logging unresolved inputs. We clarify that while this ensures execution safety, our current experiments do not evaluate robustness to prompt paraphrasing.

2. MEMOTE and biomass evaluations We added the 5-component MEMOTE weighting rule (Eq. 12, p. 7, lines 342–354). Figure 2 reports partial annotation/SBO scores, where GemAgents (median 83.9) outperforms CarveMe (69.1) and ModelSEED (43.6), approaching the published-model group (75.8). For biomass output, GemAgents (1.39) aligns most closely with curated references (0.98) compared to CarveMe (2.58) and ModelSEED (1.52) (Fig 2, pp. 7–8, lines 368–381). This establishes reference-model concordance and annotation quality, not independent experimental accuracy.

3. Reference exposure & data leakage We explicitly defined evaluation tracks (de_novo_public, biomass_controlled, reference_assisted) to rigorously track provenance and reference hashes (p. 4, lines 187–197). The BiGG benchmark and iML1515 case study use reference_assisted configurations; these controls make the use of public models inspectable rather than treating them as de novo evaluations.

4. iML1515 scaffold purpose The scaffold supplies a chemistry-preserving prior, mapping GPR via exact sequence-hash matches (p. 8, lines 385–397). With 1,497 sequence matches (out of 1,516 records), high architectural retention is expected. GemAgents deterministically replicates archived growth: 0.8774 vs 0.8770 h⁻¹ (aerobic) and 0.1620 vs 0.1575 h⁻¹ (anaerobic) (Table 1, p. 8, lines 392–418). This characterizes reference-assisted reconstruction capabilities.

5. Concrete natural-language example Appendix C.1 (Fig 5, p. 18, lines 922–949; referenced on p. 4, lines 185–186) now illustrates the full pipeline (intent →\to validation →\to artifact generation) for a specific request: "Build a bacterial GEM from proteins.faa; use aerobic glucose-minimal medium, preserve provenance, run QC, and export SBML."

6. Input data and prior knowledge We expanded the input specifications (p. 4, lines 213–215; p. 5, lines 216–220) to detail supported assets (FASTA/FNA, PGAP/NCBI, HMM, CLEAN, BiGG/ModelSEED, templates) and provenance tracking metadata (DB versions, thresholds, ambiguity) (p. 4, lines 201–212).

Thank you for the response. While this additional information is useful to some extent, I don't think it directly or fully addresses the major concerns I raised in my original review. The work has potential, but it believe it would require a more thorough revision to address the concerns satisfactorily. I will maintain the original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.