- Rating
- 1
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
GemAgents is an agentic framework that uses an LLM to interpret natural-language requests and orchestrate deterministic tools for genome-scale metabolic model reconstruction and analysis. It combines evidence-aware reconstruction, weighted gap filling, model auditing and counterexample-guided repair, and evaluates the resulting models against CarveMe, ModelSEED, and published references, with additional iML1515 and strain-design case studies.
Critical strengths
The paper presents a well-motivated separation between LLM-based orchestration and deterministic scientific computation. By constraining agent actions through explicit contracts, provenance tracking, and verification/repair modules, the framework reduces the risk that language-model errors directly propagate into scientific conclusions.
Critical weaknesses
The manuscript contains multiple unresolved internal inconsistencies that undermine confidence in the reported results. The most serious example is Figure 2: In result section, the author claims "GemAgents achieves a median score of 83.9, compared to 69.1 for CarveMe and 43.6 for ModelSEED Figure2". However, the actual Figure 2 is only a schematic of the GemAgents workflow and contains no benchmark data. Besides, in section 4.3, the author claims "For example, Flux Variability Analysis(FVA) ..." and Figure 4 is described as a succinate FSEOF analysis, while the plotted result is totally different. The manuscript also contains unresolved references such as “Appendix ??” and “(??)” in Algorithm 2.
Questions for authors
Could the authors clarify the actual content of Figure 2 and provide the missing benchmark results referenced in Section 4.1, including sample size and variability across reconstructed models?
Since most quantitative evaluations focus on the reconstructed metabolic models, could the authors clarify how the LLM-based agentic component itself was evaluated, and what additional capability it provides beyond the deterministic reconstruction and analysis pipeline?
2 comments
Thank you for the corrections to Figures 2 and 4 and the unresolved references. These fix some presentation issues, but they do not address my main concerns about the missing benchmark details and the unclear evaluation of the LLM-based agentic component. I therefore maintain my original rating.
Thank you for your careful reading and for identifying these inconsistencies. We apologize for the confusion they caused. We have corrected the figure descriptions and unresolved internal references. The corresponding revisions are highlighted in blue in the revised manuscript.
Figure 2 and Section 4.1. Figure 2 now presents the benchmark comparison: panel (a) shows the distributions of MEMOTE aggregate scores for CarveMe, ModelSEED, GemAgents, and the published-model group; panel (b) shows the corresponding predicted biomass objective outputs. The benchmark statement reports median scores of 83.9, 69.1, 43.6, and 75.8, respectively. The revised statement and caption are highlighted in blue on page 7, lines 352–354 and 368–374. The workflow schematic is Figure 1.
Figure 4 and Section 4.3. We revised Section 4.3 and the Figure 4 caption to describe the lycopene FSEOF case, replacing the earlier description of succinate flux traces. Figure 4 displays published lycopene titers from Choi et al. (2010) for the dxs background and additional gene-amplification combinations. The revised text distinguishes this figure from the methodological description of FVA in Section 3.5. These revisions are highlighted in blue on page 9, lines 434–442, 457–462, and 464–470.
Unresolved references in the Discussion. We replaced “Appendix ??” with explicit references to Section 4.3 and Figure 4 for the FSEOF workflow and Appendix C.2 for the production-envelope analysis. The corrected sentence is highlighted in blue on page 10, lines 511–512.
Unresolved references in Algorithm 2. The two “(??)” references now resolve to “Equation (7)“ for parsimonious witness extraction and “Equation (9)“ for directional bound restrictions. The corrected algorithm lines are highlighted in blue on page 17, lines 888 and 892; the corresponding equations are highlighted on page 6, lines 271–275 and line 285.
We also checked the complete manuscript and recompiled it to confirm that no undefined citations or internal cross-references remain. We appreciate your feedback, which helped us improve the manuscript’s clarity and consistency.