acceptodds
Under review as a conference paper at ICLR 2027

ToolMol: Evolutionary Agentic Framework for Multi-objective Drug Discovery

Abstract

Advances in large language models (LLMs) have recently opened new and promising avenues for small-molecule drug discovery. Yet existing LLM-based approaches for molecular generation often suffer from high rates of invalid and low-quality ligand candidates, a result of the syntactic limitations of current models with regard to molecular strings. In this paper, we introduce , an evolutionary agentic framework for de novo drug design. combines a multi-objective genetic algorithm with an agentic LLM operator that iteratively updates the ligand population. We build a comprehensive toolbox of RDKit-backed functions that allows our agentic operator to consistently make precise ligand modifications. achieves state-of-the-art performance on multi-objective property optimization tasks, discovering drug-like and synthesizable ligands that have >10% stronger predicted binding affinity compared to existing methods, evaluated on three protein targets. ligands additionally achieve state-of-the-art results in gold-standard Absolute Binding Free Energy scores, gaining over existing methods by over 35%. By studying chain-of-thought reasoning traces, we observe that tool-calling enables the model to more faithfully execute its planned modifications, efficiently exploiting the strong chemical prior knowledge in LLMs.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Drug Discovery, Large Language Models, Tool calling, Evolutionary Algorithms

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_m7VQOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce ToolMol - a genetic algorithm incorporating LLM-agent that is used to generate new candidates based on parent structures. LLM-agent has access to several RDKit-based tool for different molecular modifications which prevents agent from generating invalid and low-quality SMILES strings. The authors demonstrate that ToolMol outperforms several existing approaches in three biding affinity, QED and SA-based metrics. ToolMol is also shown to surpass MF-LAL in ABFE scores across 2 targets.

Critical strengths
  1. The paper addresses an important limitation of LLMs as molecule generators. The idea to supply LLM with RDKit-based tools is interesting.
  2. The authors perform computationally expensive ABFE simulations. Although the comparison is limited this provides additional evidence that the generated molecules can achieve favorable binding-affinity estimates under a different computational evaluation method.
Critical weaknesses

o1. My main concern is that the same scores (binding affinity, QED, SA) are used both as optimization objectives and comparison criteria in Table 1. This introduces a risk of oracle overoptimization meaning that ToolMol might become particularly good at discovering molecules that e.g. Boltz-2 systematically overestimates. If the authors added ABFE evaluation for all methods in Table 1, ToolMol would not necessarily remain the best-performing method, so it requires clarification. 2. The reported ABFE improvements are substantially smaller after applying drug-likeness and synthesizability filters. In particular, the filtered improvements over MF-LAL are only 0.6 kcal/mol for c-MET and 0.2 kcal/mol for BRD4, against considerable variation in the reported scores. The evidence does not establish that these differences are statistically significant. 3. It remains unclear whether the reported gains in ToolMol's performance arise from the LLM's chemical reasoning or simply from the constrained RDKit action space and deterministic molecular editing operations. What if we remove LLM and randomly apply the tools to molecules? 4. One of the main motivations for ToolMol is the high rate of invalid molecules generated by LLMs. However, it is unclear whether the proposed multi-step tool-calling procedure is more computationally efficient than simply generating molecules directly with an LLM and discarding invalid outputs. ToolMol allows up to 10 sequential tool-calling steps per molecule, potentially requiring substantially more LLM inference than direct molecular generation. The paper does not provide a comparison of the total inference cost, number of LLM calls, or wall-clock time required to obtain a fixed number of valid molecules. 5. The evaluation covers only three protein targets which can be insufficient because it may not represent the diversity of binding-pocket geometries and target classes encountered in drug discovery.

Questions for authors
  1. Why are ABFE results in Table 2 restricted to MF-LAL and other baselines are excluded? Why is ACAA1 excluded from Table 2? Could the authors provide ABFE results for at least top-3/5 molecules generated by other baselines?
  2. Could the authors report the average number of LLM calls, generated tokens, and inference time required per valid molecule? How does ToolMol compare against direct generation with validity filtering under a matched computational budget?
  3. Could the authors compare ToolMol against a random algorithm that applies exactly the same molecular editing tools within the same genetic algorithm and oracle budget?
  4. In Table 2, the ABFE improvements of filtered ToolMol molecules over MF-LAL are relatively small compared to the reported standard deviations. Are these differences statistically significant? Could the authors report confidence intervals or statistical tests to better support the claimed improvements?
8 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/2)

My main concern is that the same scores (binding affinity, QED, SA) are used both as optimization objectives and comparison criteria in Table 1. This introduces a risk of oracle overoptimization meaning that ToolMol might become particularly good at discovering molecules that e.g. Boltz-2 systematically overestimates. If the authors added ABFE evaluation for all methods in Table 1, ToolMol would not necessarily remain the best-performing method, so it requires clarification.

We acknowledge the reviewer’s concern about limited evaluation. Due to the extremely expensive cost of ABFE, we are unable to report ABFE evaluation for all methods in Table 1. However, we were able to evaluate molecules generated by MOLLEO, the closest methodological neighbor and most competitive baseline. We ran experiments on the top 15 molecules from the filtered sets for MOLLEO on c-MET and BRD4, consistent with the existing ABFE experiments shown in Table 2. For MOLLEO on c-MET, we obtain ABFE predicted binding affinity of −6.57±4.17-6.57 \pm 4.17, with corresponding average QED and SA of 0.60±0.080.60 \pm 0.08 and 2.53±0.172.53 \pm 0.17 respectively. For MOLLEO on BRD4, we obtain ABFE predicted binding affinity of −6.40±4.50-6.40 \pm 4.50, with corresponding average QED and SA of 0.61±0.110.61 \pm 0.11 and 2.63±0.282.63 \pm 0.28 respectively. Contrasting these results with ToolMol (filtered) and MF-LAL in Table 2, we note that MOLLEO's c-MET affinity result is not only significantly worse than ToolMol, but is also worse than the MF-LAL baseline. MOLLEO's BRD4 affinity result is also weaker than ToolMol’s, and demonstrates weaker QED (but slightly stronger SA). Altogether, we believe that this additional ABFE result demonstrates that the Boltz-2 scores in Table 1 have good correlation with ABFE scores, and that the high ABFE scores achieved by ToolMol over multiple existing baselines are non-trivial and non-coincidental.

As an aside, the reviewer’s concern about overoptimizing for Boltz-2 is understandable, but we would like to reference a recent study [1] in which candidates filtered by Boltz-2 predicted affinity were experimentally verified in a wet-lab. Candidate compounds that scored highly on Boltz-2 reported a 70% success rate in inhibitory activity. We would like to note that we chose a more expensive oracle in Boltz-2 over cheaper oracles such as Vina exactly for the benefit of relying on an oracle that has demonstrated real-world success, so that we are less concerned about overoptimizing for this expensive oracle. We would also like to note that much well-regarded literature in this field trains their generative method on CrossDocked2020 and evaluates on the CrossDocked2020 test set, using Vina both as a training signal and as the evaluation metric. Our pattern of relying on Boltz-2 in optimization and in evaluation is fundamentally the same.

[1] Kawano, Shinya, et al. “Efficient Discovery of Potent AKR1B10 Inhibitors Using Co-folding Model Boltz-2.” Journal of Chemical Information and Modeling, vol. 66, no. 18, 2026, pp. 12100–14, https://doi.org/10.1021/acs.jcim.6c00874.

The reported ABFE improvements are substantially smaller after applying drug-likeness and synthesizability filters

We acknowledge that our gains on the filtered sets are not statistically significant, but would like to clarify that it was never the intent/focus of this work to achieve state-of-the-art in ABFE. We included these evaluations as a supporting result alongside Boltz-2, and the most surprising/important result from our manuscript is that we are able to match and slightly exceed the ABFE results of MF-LAL, which is a method explicitly tuned for this oracle that includes explicit ABFE signal within its active learning pipeline. ToolMol receives zero ABFE guidance and has no knowledge of this oracle whatsoever, and yet yields such strong results simply by optimizing Boltz-2 predicted affinity with a tool-assisted LLM. This is the only result and idea that we wanted to raise in section 4.2.

The paper does not provide a comparison of the total inference cost, number of LLM calls, or wall-clock time required to obtain a fixed number of valid molecules.

We find that the average number of LLM prompts per generated molecule is around 5.45.4 in ToolMol, compared to a single call for MOLLEO. However, we observe that since compute is generally bottlenecked by the Boltz-2 evaluation rather than LLM inference (which is widely available and supported by massive infrastructure in the modern era), the total actual runtime of ToolMol is negligibly greater than that of MOLLEO.

AuthorsOpenReview ↗

Rebuttal by Authors (2/2)

The evaluation covers only three protein targets which can be insufficient because it may not represent the diversity of binding-pocket geometries and target classes encountered in drug discovery.

While we understand this concern, we raise that our protein targets were intently chosen with structural and functional diversity in mind. c-MET is a receptor tyrosine kinase, BRD4 is a bromodomain-containing protein, and ACAA1 is a peroxisomal thiolase. We believe that ToolMol’s consistent performance across these structurally diverse binding sites reflects a genuine methodological advancement rather than coincidence, given that these three handpicked binding tasks reflect unique chemical functions, structures, and properties. We would also like to note that many published, established work in this field evaluate on a similar number of protein targets. MOLLEO evaluates on 3 targets, LIMO evaluates 2, GENTRL focuses on a single target (DDR1) and achieves fantastic results on it. Our work focuses on slower but more accurate oracles such as Boltz-2 and ABFE, as this reflects higher potential for real-world applicability, which is always the ultimate goal of computational drug discovery. It is our belief that while this restricts feasible evaluations to fewer targets, our careful selection of targets and higher fidelity evaluations provides results that are still important and impactful.

Why are ABFE results in Table 2 restricted to MF-LAL and other baselines are excluded? Why is ACAA1 excluded from Table 2?

ABFE is extremely expensive to evaluate, and as such we did not have the compute or time to evaluate on every single baseline found in Table 1. We hope that the added MOLLEO result above is helpful and informative. ACAA1 is excluded because the primary baseline, MF-LAL, is only evaluated on c-MET and BRD4. Evaluating on ACAA1 would require reproducing results with the MF-LAL method, which involves computing further ABFE signals within their multi-fidelity active training pipeline, which was also outside the scope of our compute budget.

How does ToolMol compare against direct generation with validity filtering under a matched computational budget?

This ablation already exists in Table 3 of section 4.3 of our manuscript; we ran MOLLEO but allowed the LLM to retry invalid generations up to 10 times, ensuring that every molecule in the final generated set was LLM-produced. As shown by Table 3, this direct generation scheme is significantly outperformed by ToolMol.

Could the authors compare ToolMol against a random algorithm that applies exactly the same molecular editing tools within the same genetic algorithm and oracle budget?

We thank the reviewer for suggesting this valuable ablation, and have designed a random version of ToolMol, where we reuse the exact same seven tools and input random constrained parameters that are guaranteed to be valid. The results with the multi-objective metrics from Table 1 (filtered affinity, hypervolume) are as follows:

ToolMol (random)ToolMol (LLM)
c-MET FA−9.31±0.14-9.31 \pm 0.14−10.35±0.17\mathbf{-10.35 \pm 0.17}
c-MET HV0.57±0.0070.57 \pm 0.0070.62±0.01\mathbf{0.62 \pm 0.01}
BRD4 FA−9.08±0.02-9.08 \pm 0.02−9.91±0.18\mathbf{-9.91 \pm 0.18}
BRD4 HV0.56±0.0060.56 \pm 0.0060.60±0.01\mathbf{0.60 \pm 0.01}
ACAA1 FA−8.30±0.04-8.30 \pm 0.04−8.78±0.15\mathbf{-8.78 \pm 0.15}
ACAA1 HV0.52±0.010.52 \pm 0.010.54±0.008\mathbf{0.54 \pm 0.008}

The random tool-use ablation is significantly outperformed by ToolMol guided by LLM reasoning, showing that the LLM operator provides valuable decision-making within the design process. These random tools yield similar results to Graph-GA in Table 1 (average rank of 5.0), which makes sense since Graph-GA is also a random rule-based operator.

Thank you for your clarifications and additional results. I appreciate that ABFE calculations are computationally expensive, and I acknowledge the evidence supporting Boltz-2 as a useful proxy for real-world binding affinity. I am also satisfied with ToolMol (random) results and clarifications regarding the number of LLM calls. However, my main concern (point 1 in "Critical Weaknesses") remains unresolved. In Table 1, ToolMol explicitly optimizes binding affinity, QED, and SA, and is subsequently evaluated using these same objectives. While this is appropriate for assessing optimization performance, the comparison may favor methods that explicitly optimize these objectives over methods that do not have access to the same iterative optimization procedure or oracle feedback (Pocket2Mol, TAGMol, and PAFlow). I understand that you provide ABFE results as an independent molecules assessment, but you do it only for MF-LAL (which is not included in Table 1 for some reason) and for MOLLEO, leaving the performance of the other baselines under an independent evaluation unclear. Given that ABFE is computationally expensive, you could think of other possible independent metrics that can support results reported in Table 1, e.g. QSAR IC50 predictors, medicinal chemistry experts assessments, alternative synthetic accessibility scores (RAscore) etc.

Reviewer_jhfYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ToolMol, an LLM-based evolutionary framework for multi-objective molecular optimization. Instead of directly generating SMILES strings, the LLM uses seven RDKit-backed tools to perform molecular crossover and mutation within a genetic algorithm. Experiments on three protein targets show improvements in filtered binding affinity and hypervolume over existing methods. The paper also provides ABFE evaluations and ablation studies to analyze the effectiveness of tool calling.

Critical strengths
  1. The paper is well motivated, and using RDKit-backed tools instead of direct SMILES generation provides a practical way to reduce invalid molecular edits.
  2. ToolMol achieves the best filtered affinity and hypervolume across all three evaluated protein targets, consistently outperforming MOLLEO.
  3. The ablation studies help isolate the contribution of tool calling and show that the toolbox also improves performance when integrated into ShinkaEvolve.
  4. The additional ABFE evaluations and reasoning-trace analysis provide useful complementary evidence beyond the main optimization results.
Critical weaknesses
  1. Limited evaluation across protein targets. The experiments cover only three proteins, making it difficult to assess the generalizability of ToolMol. The results resemble a few target-specific case studies rather than a systematic evaluation across diverse protein targets.
  2. Limited evidence from ABFE evaluation. The large ABFE gains are mainly observed for unfiltered molecules with poor average QED scores. After QED/SA filtering, the improvements over MF-LAL become substantially smaller, particularly for BRD4 (-6.4 vs. -6.2 kcal/mol). The large reported variability further weakens the evidence for substantial ABFE improvements.
  3. Reliance on Boltz-2 for optimization and evaluation. The same affinity predictor is used to guide molecular optimization and evaluate the main results, raising concerns about predictor-specific biases. Further evaluation of the generated molecules using independent scoring functions (e.g., AutoDock Vina or GNINA) would help establish whether the improvements generalize beyond Boltz-2.
Questions for authors
  1. Could the authors compare ToolMol with random or heuristic selection of the same seven RDKit editing tools to better isolate the contribution of LLM reasoning?
  2. Could the authors report the average number of LLM/tool calls per generated molecule and the overall runtime or inference cost compared with MOLLEO?
4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for their valuable feedback and insight about our work. We respond to the concerns raised by the reviewer below:

Limited evaluation across protein targets. The experiments cover three proteins, making it difficult to assess the generalizability of ToolMol.

While we understand this concern, we raise that our protein targets were intently chosen with structural and functional diversity in mind. c-MET is a receptor tyrosine kinase, BRD4 is a bromodomain-containing protein, and ACAA1 is a peroxisomal thiolase. We believe that ToolMol’s consistent performance across these structurally diverse binding sites reflects a genuine methodological advancement rather than coincidence, given that these three handpicked binding tasks reflect unique chemical functions, structures, and properties. We would also like to note that many published, established work in this field evaluate on a similar number of protein targets. MOLLEO evaluates on 3 targets, LIMO evaluates 2, GENTRL focuses on a single target (DDR1) and achieves fantastic results on it. Our work focuses on slower but more accurate oracles such as Boltz-2 and ABFE, as this reflects higher potential for real-world applicability, which is always the ultimate goal of computational drug discovery. It is our belief that while this restricts feasible evaluations to fewer targets, our careful selection of targets and higher fidelity evaluations provides results that are still important and impactful.

Limited evidence from ABFE evaluation

We acknowledge that our gains on the filtered sets are not statistically significant, but would like to clarify that it was never the focus of this work to achieve SOTA in ABFE. We included these evaluations as a supporting result alongside Boltz-2, and the most surprising result from our manuscript is that we are able to match and slightly exceed MF-LAL, which is a method explicitly tuned for this oracle that includes explicit ABFE signal within its active learning pipeline. ToolMol receives zero ABFE guidance and has no knowledge of this oracle whatsoever, and yet yields such strong results simply by optimizing Boltz-2 predicted affinity with a tool-assisted LLM. This is the only result and idea that we wanted to raise in section 4.2.

Reliance on Boltz-2 for optimization and evaluation

We recognize the reviewer’s concern. However, we don’t fully agree that including e.g. Vina would be a good secondary evaluation. The point of a search algorithm such as ToolMol or MOLLEO is to successfully optimize for the objective(s) that guide the search, and as such we evaluate the results of the algorithm on those objectives to see how well the search satisfied that criteria. Evaluating on Vina (which is also known to be less accurate than Boltz-2) provides little useful information about the methodological success of these methods. Much well-regarded literature in this field trains their generative method on CrossDocked2020 then evaluates on the CrossDocked2020 test set, using Vina both as a training signal and as the evaluation metric. Using Boltz-2 in optimization and in evaluation is fundamentally the same. Finally, we would like to note that ABFE (a more accurate oracle) already provides a valuable secondary evaluation.

Could the authors compare ToolMol with random or heuristic selection of the same seven RDKit editing tools to better isolate the contribution of LLM reasoning?

We thank the reviewer for suggesting this valuable ablation, and have designed a random version of ToolMol, where we reuse the exact same seven tools and input random constrained parameters that are guaranteed to be valid. The results with the multi-objective metrics from Table 1 (filtered affinity, hypervolume) are as follows:

ToolMol (random)ToolMol (LLM)
c-MET FA−9.31±0.14-9.31 \pm 0.14−10.35±0.17\mathbf{-10.35 \pm 0.17}
c-MET HV0.57±0.0070.57 \pm 0.0070.62±0.01\mathbf{0.62 \pm 0.01}
BRD4 FA−9.08±0.02-9.08 \pm 0.02−9.91±0.18\mathbf{-9.91 \pm 0.18}
BRD4 HV0.56±0.0060.56 \pm 0.0060.60±0.01\mathbf{0.60 \pm 0.01}
ACAA1 FA−8.30±0.04-8.30 \pm 0.04−8.78±0.15\mathbf{-8.78 \pm 0.15}
ACAA1 HV0.52±0.010.52 \pm 0.010.54±0.008\mathbf{0.54 \pm 0.008}

The random tool-use ablation is outperformed by ToolMol, showing that the LLM operator provides valuable decision-making within the design process. These random tools yield similar results to Graph-GA in Table 1 (average rank of 5.0).

Could the authors report the average number of LLM/tool calls per generated molecule and the overall runtime or inference cost compared with MOLLEO?

We find that the average number of LLM prompts per generated molecule is around 5.45.4 in ToolMol, compared to a single call for MOLLEO. However, we observe that since compute is generally bottlenecked by the Boltz-2 evaluation rather than LLM inference (which is widely available and supported by massive infrastructure in the modern era), the total actual runtime of ToolMol is negligibly greater than that of MOLLEO.

Thank you for the detailed responses and additional experiments. The random tool-use ablation (Q1) helps clarify the contribution of LLM-guided decisions, while the additional MolSkill and RAScore evaluations partially address my concern about relying on the same metrics for optimization and evaluation (W3). The clarifications regarding inference overhead (Q2) are also helpful.

However, two concerns remain: the limited evaluation across only three protein targets (W1), and the relatively small and statistically insignificant improvements in filtered ABFE results (W2), which limit the evidence for consistent binding affinity improvements under independent evaluation. Given these remaining limitations, I will maintain my original rating.

We graciously thank the reviewer for their time, effort, and consideration. We maintain that evaluating across three protein targets is very common within this field and that our target selection is structurally and functionally broad. Any further evaluation introduces computational overhead that we were not able to support, given that all evaluations are done with the relatively expensive Boltz-2. We additionally maintain that it was never the focus of this work to achieve SOTA in ABFE; the ABFE results are secondary to Table 1, and that the fact that ToolMol even matches the ABFE-specialized MF-LAL is already a very suprising result. We understand and accept that the reviewer disagrees with these arguments, but we kindly ask the AC and any other reviewing body to closely consider these 2 particular points that is holding the reviewer from modifying their score.

Thank you for the further clarification. I understand the authors' arguments and appreciate their efforts. In my assessment, the paper falls between a 2 and a 3 (roughly 2.5). While I will maintain my current rating, I recognize the merits of this work and would be supportive of its acceptance if the AC considers it a borderline case.

Reviewer_Yd2fOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes to use an LLM-based update rule for evolutionary multi-property optimization of molecules. Using RDKit tools in the LLM update rule allows ligand changes to be made more precisely and accurately (less syntactically invalid strings), thus improving the overall success of the framework. The framework achieves state of the art results compared to a number of strong baselines.

Critical strengths
  1. The overall idea of using the tool-calling functionality of LLMs is simple and effective. While the idea presented is nothing ground-breaking, it is a strong positive that this paper establishes tool-calling as a solid foundation to build other "LLM for science" ideas on where certain guarantees need to be made, such as the case here for molecular strings. Specifically, the application in genetic algorithms which are applicable to many domains makes the idea interesting and worthy of acceptance for me.
  2. This is supported by reasonably thorough experimental evaluation, ablations, and nuanced discussion of limitations, such as the Boltz-2 metric. This makes the paper interesting, independently from the first strength, for researchers in this domain. I appreciate the case study of a concrete example in Appendix A, demonstrating how the tool calling changes the nature of the reasoning trace.
Critical weaknesses
  1. Isolation of LLM capabilities: The paper limits its analysis to a single LLM, GPT-OSS-120B. My concern is that this makes some of the conclusions specific to this LLM that may not carry forward to others, e.g. other models may have better or worse tool calling abilities in this domain, or are better able to act as an AgentGen step with lower failure rate and without the need for explicit tools. It would have been good to either benchmark these capabilities for a few, or at least point to a reference that justifies this choice for this domain. This is the main point that makes me hesitant on fully recommending this paper for acceptance.
Questions for authors
  1. [Suggestion for future work] One of the primary motivations for using tools seems to be to avoid the generation of invalid SMILES strings. However, that puts into question if SMILES is the correct representation in the first place. By using tools, it seems like you are already abstracting away some of the mistakes-prone SMILES interactions. Have you considered reasoning over a molecule representation like graphs that are perhaps less likely to run into SMILES string validity issues?
3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for their valuable feedback and insight about our work. We respond to the concerns raised by the reviewer below:

Isolation of LLM capabilities: The paper limits its analysis to a single LLM, GPT-OSS-120B. My concern is that this makes some of the conclusions specific to this LLM that may not carry forward to others, e.g. other models may have better or worse tool calling abilities in this domain, or are better able to act as an AgentGen step with lower failure rate and without the need for explicit tools. It would have been good to either benchmark these capabilities for a few, or at least point to a reference that justifies this choice for this domain. This is the main point that makes me hesitant on fully recommending this paper for acceptance.

While we completely understand and appreciate this concern, and have prepared additional experimental results for the reviewer’s consideration, we would initially like to raise that while other frontier models may have better syntactic success without the use of tools, the tackled problem in this paper is not primarily poor SMILES validity. Rather, we aim to improve the quality of LLM-generated compounds as a whole, and syntactic validity was only one part of this problem. That being said, we agree that benchmarking on current frontier models is valuable; for your consideration, we ran GPT-5.6-Sol, Qwen-3.8-Flash-Next, and DeepSeek-V4.1-Flash on the MOLLEO (no tools) prompt for 300 iterations on distinct parent ligand pairs. Out of 300 independent generations, these 3 models still yielded a total of 10, 15, and 5 invalid SMILES respectively. While this failure rate is lower than that of GPT-OSS-120B, it is still much more than what would be observed by using tools with any model. Over the course of a full genetic algorithm run on our oracle budget, we could expect these frontier models to generate over 20-40 invalid ligands, which could greatly hinder results. ToolMol consistently generates 0 invalid ligands over the entire optimization loop. Unfortunately due to budget, compute, and time constraints, we are not able to rigorously benchmark these frontier models on all 3 protein targets over multiple seeds and multiple methods during this brief discussion period, but we hope that this study of syntactic validity provides some that evidence that current frontier models would still greatly benefit from access to deterministic tools.

Furthermore, beyond just syntactic invalidity, we also have significant evidence to support that tool-calling improves the inherent quality of the generated compounds themselves; Table 3 in the manuscript provides an ablation where we allow MOLLEO to retry a generation if an invalid generation is produced by the LLM; this cuts the total number of non-LLM generated compounds in the final ligand set to near-zero, essentially removing the impact of invalid syntax. In this environment, MOLLEO is still convincingly beaten by ToolMol across the board. We believe this shows that the tool-calling process has significant benefits beyond just reducing the syntactical failure surface; it demonstrably improves the inherent quality of generated compounds altogether. Because of this, we believe that even as base LLMs improve, the integration of a molecular toolbox will always provide a pure advantage in the generation process, allowing any model to focus more on the design choices for ligand generation rather than on the syntactic sugar. This is particularly relevant given that modern agentic workflows have largely been governed by the ability of LLMs to use a large array of tools, not just pure reasoning through problems in the token space.

Have you considered reasoning over a molecule representation like graphs that are perhaps less likely to run into SMILES string validity issues?

This is a great suggestion and just so happens to be what we plan to work on next! We agree that SMILES is indeed a limited modality for molecular understanding, and LLMs are more likely to gain more insight about a structure given a richer representation. In particular, we are experimenting with the idea of encoding a full predicted ligand-protein binding conformation (either latently through GNNs or through a textual paradigm) as part of the input to the LLM operator within a search algorithm. The goal is to ground the LLM in physical interactions to give it the best contextual understanding of the optimization problem. This is a good and natural next step, but we hope that the ideas presented in this current work are also interesting to the reviewer.

We thank the reviewer for their consideration and useful comments.

Thank you for the response and the preliminary investigation into using other LLMs, as well as clarifying the intended positioning. I remain positive in my evaluation of this paper, and I did not see any issues that I consider major that the other reviewers raised.

We graciously thank the reviewer for their time, effort, and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.