acceptodds
Under review as a conference paper at ICLR 2027

SMDD-Bench: Can LLMs Solve Real-World Small Molecule Drug Design Tasks?

Abstract

LLM agents have incredible potential for scientific discovery applications. However, the performance of LLM agents on real-world, small molecule drug design (SMDD) tasks across diverse chemistries and targets is unclear. Current evaluation methods are either ad hoc, too simple for real-world discovery, limited in scale, or restricted to single-turn question answering. In effort to standardize the evaluation of LLM agents on small molecule design, we introduce SMDD-Bench, a challenging, multi-turn, long-horizon agentic benchmark consisting of 502 guaranteed-solvable task instances spanning 5 task types: 2D Pharmacophore Identification, Interaction Point Discovery, Scaffold Hopping, Lead Optimization, and Fragment Assembly. SMDD-Bench tasks span a wide region of chemical space and involve 102 unique protein targets. Completely solving the benchmark would require having strong chemical and biological reasoning and 3D intuition, understanding specialized tool use, and displaying planning expertise over a limited number of oracle calls. We benchmark 7 frontier open and closed source LLMs and find even the most performant LLM, GPT5.4, solves only 40.2% of tasks. We hope SMDD-Bench provides a standardized testbed to invigorate the field towards training and evaluating LLM agents for fully autonomous computational drug design.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
LLM agents, small molecule drug discovery

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_QygtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SMDD-Bench, a benchmark of 502 small-molecule drug design tasks spanning five task types and 102 protein targets. Agents work through a multi-turn tool interface with limited access to Boltz2 and ADMET-AI. A central contribution is witness-aware task generation: a hidden molecule satisfying the evaluation criteria is used to construct each scaffold hopping, lead optimization, or fragment assembly instance. Seven LLMs are evaluated. The best overall success rate is 40.2%, with most successes coming from lead optimization. Additional analyses examine output diversity, candidate selection, and failures across reasoning trajectories.

Critical strengths

(1)Constructing tasks around a hidden feasible molecule gives the evaluation a firmer basis. This is particularly useful for lead optimization and fragment assembly, where satisfying several requirements simultaneously is central to the task. (2)The paper goes beyond reporting model rankings.

Critical weaknesses

(1)The connection between benchmark success and real-world design quality remains insufficiently established. An independent assessment of a representative subset of witnesses and successful submissions would help establish whether the benchmark rewards chemically meaningful improvements rather than predictor-specific behavior. (2)The current baselines do not adequately calibrate task difficulty or support a broad interpretation of the low success rates.

Questions for authors

(1)Can the authors provide an independent assessment of a representative subset of witness molecules and successful agent submissions? The key question is whether improvements defined by the benchmark remain credible outside Boltz2 and ADMET-AI. (2)For scaffold hopping and fragment assembly, what proportion of submissions fail at each evaluation gate? (3)How should the scaffold hopping retrieval result of 5.8% in Table 16 be reconciled with the statement that non-LLM baselines do not outperform LLMs? (4) How stable are the reported model differences across repeated runs? The overall scores of GPT-5.4, Gemini, and Claude are close, while several task types contain only 25–60 instances.

3 comments
AuthorsOpenReview ↗

Author Response 1

First off, we would like to thank the reviewer for the review! Below are our point-by-point responses to your concerns.

(1) The connection between benchmark success and real-world design quality remains insufficiently established. An independent assessment of a representative subset of witnesses and successful submissions would help establish whether the benchmark rewards chemically meaningful improvements rather than predictor-specific behavior.

We had human experts validate for task validity, threshold appropriateness, constraint satisfaction, and witness evaluation against all oracles, optimization properties, hold-constant properties, and hard constraints. We will include this in the manuscript

(2) The current baselines do not adequately calibrate task difficulty or support a broad interpretation of the low success rates.

We strongly disagree with this statement. Small molecule drug design is a very challenging field, for both humans and LLMs. The low success rates of baseline methods calibrate how much lift LLMs can achieve beyond simple mechanisms of understanding the task.

(1) Can the authors provide an independent assessment of a representative subset of witness molecules and successful agent submissions? The key question is whether improvements defined by the benchmark remain credible outside Boltz2 and ADMET-AI.

We think there may have been a misunderstanding here – the witness molecules are derived from experimental databases such as ChEMBL. These molecules have already been lab validated at high fidelity and are then re-validated under the SMDD-Bench computational oracles. The existence of the witness molecule ensures the existence of a successful agent submission. The relationship between benchmark oracles and wet lab labels has already been validated by [1] and [2].

[1] Boltz-2: Towards Accurate and Efficient Binding Affinity Prediction, https://www.biorxiv.org/content/10.1101/2025.06.14.659707v1 [2] ADMET-AI: a machine learning ADMET platform for evaluation of large-scale chemical libraries, https://pmc.ncbi.nlm.nih.gov/articles/PMC11226862/

(2) For scaffold hopping and fragment assembly, what proportion of submissions fail at each evaluation gate?

Below are tables with the proportion of overall submissions which fail at each gate for the scaffold hopping and fragment assembly tasks:

Scaffold Hopping:

#GateReachedFailed Here
0Validity100.0%0.0%
1Tanimoto Similarity100.0%0.3%
2Scaffold MCS99.7%14.3%
3Boltz Interaction Similarity85.4%82.0%
4Boltz Binding Probability3.4%0.3%
—All gates passed3.1%—

Fragment Assembly:

#GateReachedFailed Here
0Validity100.0%2.0%
1Fragment Incorporation98.0%19.8%
2Fragment Linking78.2%0.9%
3Hard Constraints77.3%18.5%
4Boltz Binding & Pose58.8%58.1%
—All gates passed0.7%—

(3) How should the scaffold hopping retrieval result of 5.8% in Table 16 be reconciled with the statement that non-LLM baselines do not outperform LLMs?

The scaffold hopping retrieval baseline is retrieval and sorting over the experimental potency values from the ChEMBL database itself, which gives the retrieval baseline highly privileged information. The reasoning behind including this baseline was to measure if LLM performance came from memorizing the ChEMBL database – to which we find this is not the case.

(4) How stable are the reported model differences across repeated runs? The overall scores of GPT-5.4, Gemini, and Claude are close, while several task types contain only 25–60 instances.

We performed 20 repeated runs of specific tasks for the selected tasks in the SMDD-Bench Diversity experiment. Below are the results of those runs:

ModelAccuracy
Claude Sonnet 4.684.0 ± 11.0
GPT-5.479.0 ± 7.4
Gemini 3.1 Pro80.0 ± 10.5
Kimi K2.5 Thinking60.0 ± 12.9
DeepSeek V3.253.5 ± 10.3
MiniMax M2.760.0 ± 10.0
Qwen3.5 397B A17B72.5 ± 10.6

As seen in the table, the standard deviation is quite tight.

Once again, we would like to thank the reviewer for the reviews, and we hope we were able to address all of your concerns!

Thank you for the detailed response. The breakdown of failures by evaluation stage and the clarification of the retrieval baseline’s access to experimental data address some of my questions. My main concern remains whether the improvements observed in successful agent submissions hold up under independent evaluation beyond Boltz2 and ADMET-AI. The repeated runs on selected tasks also leave open whether the small differences among the leading models are stable across the full benchmark. Given the evidence in the current manuscript and response, my concerns are only partially resolved.Therefore, I will maintain my original rating.

Thank you for the response!

Our ultimate goal is to understand current model capabilities in small molecule drug design tasks as opposed to precisely differentiating the performance of leading models. However, we are currently running multiple runs of each model to get a notion of spread and share our early results on GPT5.4 and Claude:

GPT5.4

Task typeSD
2D pharmacophore0.00
Interaction point discovery0.00
Scaffold hopping2.36
Lead optimization1.39
Fragment assembly2.14
Overall0.43

Claude 4.6 Sonnet

Task typeSD
2D pharmacophore0.00
Interaction point discovery0.00
Scaffold hopping1.86
Lead optimization1.20
Fragment assembly1.91
Overall1.15

The increase in task count from 20 in the SMDD-Bench Diversity split to 502 for the entire SMDD-Bench contributes to a significant reduction in the overall standard deviation. We hope this alleviates your concerns regarding the variance of the runs, and we hope this addresses your concerns regarding differentiating performance of the leading models. And once again, we are running the rest of the models to get a thorough understanding of the spread of the model's performance!

My main concern remains whether the improvements observed in successful agent submissions hold up under independent evaluation beyond Boltz2 and ADMET-AI.

We want to be very clear that the benchmark is designed such that the oracles themselves are the ground truth for reasons outlined at the end of section 6. We wanted to circle back to a comment you made in your original review:

The key question is whether improvements defined by the benchmark remain credible outside Boltz2 and ADMET-AI.

The independent assessment (at greater accuracies than Boltz2 and ADMET-AI) of such improvements can only be performed at the wet lab scale -- human experts cannot simply look at molecules and determine success or favorability. The computational chemistry community already has significant precedent in using computational oracles such as Boltz2 and ADMET-AI and treating those as ground truth due to the infeasibility of sending everything to the wet lab.

We feel that there may be some gap between your expectations as well as the prior precedent within the community that we have established. We hope that we were able to clear up this gap! Please let us know if you have any other questions.

Reviewer_mMMTOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce SMDD-Bench - a benchmark for LLM agents that consists of tasks related to small molecule drug discovery. SMDD-Bench Diversity and SMDD-Bench Lite are also released to study the diversity of LLM decisions and to encourage broader benchmark adoption, respectively. The authors benchmark 7 LLMs, discuss common failure modes and demonstrate that current LLMs do not perform well on these tasks and there is a room for improvement.

Critical strengths
  1. The quality of the benchmark is high. It spans 5 clear and practically oriented task types, incorporates a wide range of targets and a vast chemical space. Evaluation of LLM's performance on these tasks does not require human in the loop. Diversity and Lite versions are useful as well.
  2. The experimentation is rigorous. Experimental details and procedures are clearly explained. The authors discuss failure modes in detail.
Critical weaknesses
  1. The authors discuss LLM's failure modes in Section 4. It provides useful insights of the current LLMs capabilities regarding drug design tasks, but it would be more interesting to see the author's view on why these failures occur and how they could be minimized. Such insights would make the study more impactful, as they could inform future improvements in LLM-based agents for drug design, rather than merely highlighting the weaknesses of existing approaches. Also, the authors do not establish how much of the reported failure is attributable to the underlying LLM versus the agent design.
  2. The information on how benchmark tasks were created is insufficient. The paper should state if these tasks were created and validated by experts, why the tasks are considered to be practically important. Also, neither benchmark tasks/solutions nor the underlying code for this study are released at lest in the anonymous format.
  3. Tables 1,2,3 lack statistical significance information. The results should at least be formatted in the mean+- std format across several LLM calls, since LLM results are not deterministic.
  4. A significant limitation is that the benchmark evaluates drug design using predictions from Boltz2 and ADMET-AI rather than experimentally measured molecular properties. The same predictive models are used for both task generation and evaluation. Consequently, an agent can succeed by exploiting model-specific biases without necessarily discovering molecules with improved experimental properties.
  5. The benchmark is dominated by lead optimization, making the overall success rate misleading. The authors could report a macro-averaged score across task categories as a primary metric.
Questions for authors
  1. To what extent do the observed failures reflect limitations of the underlying LLMs versus limitations of the agent harness? Would stronger planning, memory, or candidate-selection mechanisms improve performance?
  2. Could the authors clarify whether domain experts were involved in designing or validating the benchmark tasks? Assuming that all tasks are practically important for small molecule drug design, is it common that they are solved using LLM agents rather than specific software?
  3. How robust are the reported results across independent runs? Could the authors report mean performance and standard deviations (or confidence intervals) over multiple runs?
  4. How confident can we be that improvements in benchmark performance correspond to meaningful improvements in real-world molecular properties rather than optimization of the surrogate models like Boltz-2 or ADMET AI?
  5. Could the authors additionally report a macro-averaged success rate across all five task types?
6 comments
AuthorsOpenReview ↗

Author Response 1

We would like to thank the reviewer for the review. We were pleased to hear that you enjoyed the high quality nature of SMDD-Bench as well as the rigorous experimentation we performed. Here are our point-by-point responses to your concerns.

The authors discuss LLM's failure modes in Section 4. It provides useful insights of the current LLMs capabilities regarding drug design tasks, but it would be more interesting to see the author's view on why these failures occur and how they could be minimized. Such insights would make the study more impactful, as they could inform future improvements in LLM-based agents for drug design, rather than merely highlighting the weaknesses of existing approaches. Also, the authors do not establish how much of the reported failure is attributable to the underlying LLM versus the agent design.

We have an extensive failure mode analysis in Section 4 (Common Failure Modes) as well as deeper analysis into several agent rollouts in Appendix J. In summary, we generally find that LLMs struggle with planning multi-step molecular edits and isolating structure activity relationships. We do believe agent design can help with some of the failure cases; but ultimately, we believe that understanding complex biochemical properties and 3D spatial relationships require a step function change in intelligence that can only be embedded into the agent via novel LLM training algorithms.

The information on how benchmark tasks were created is insufficient. The paper should state if these tasks were created and validated by experts, why the tasks are considered to be practically important. Also, neither benchmark tasks/solutions nor the underlying code for this study are released at lest in the anonymous format.

The detailed benchmark task creation procedure is already in Appendix B (pages 15-23) of the manuscript. Every task was validated by human experts for task validity, threshold appropriateness, constraint satisfaction, and witness evaluation against all oracles, optimization properties, hold-constant properties, and hard constraints. We will include this in the manuscript.

Tables 1,2,3 lack statistical significance information. The results should at least be formatted in the mean+- std format across several LLM calls, since LLM results are not deterministic.

While we do not have statistical significant information for tables 1, 2 and 3 (gathering a sufficient number of runs to sufficiently decrease the variance would require on the order of $75,000-$100,000), we do have an understanding of the non-deterministic behavior of the LLMs on SMDD-Bench through our experiment conducted on the SMDD-Bench Diversity split. The variance results are shown below:

ModelAccuracy
Claude Sonnet 4.684.0 ± 11.0
GPT-5.479.0 ± 7.4
Gemini 3.1 Pro80.0 ± 10.5
Kimi K2.5 Thinking60.0 ± 12.9
DeepSeek V3.253.5 ± 10.3
MiniMax M2.760.0 ± 10.0
Qwen3.5 397B A17B72.5 ± 10.6

These results were taken from 20 identical rollouts on the same set of tasks. The table indicates that the variance due to nondeterminism is, in fact, quite tight.

AuthorsOpenReview ↗

Author Response 2

A significant limitation is that the benchmark evaluates drug design using predictions from Boltz2 and ADMET-AI rather than experimentally measured molecular properties. The same predictive models are used for both task generation and evaluation. Consequently, an agent can succeed by exploiting model-specific biases without necessarily discovering molecules with improved experimental properties.

The way SMDD-Bench relies on strong computational surrogates such as Boltz2 and ADMET-AI is in line with the accepted norms of the field of computational small molecule drug design [1-11]. Due to the intractability of the wet lab validating each result, computational surrogates have become an accepted oracle.

[1] Guo and Schwaller, Augmented Memory: Sample-Efficient Generative Molecular Design with Reinforcement Learning, JACS Au (2024). https://doi.org/10.1021/jacsau.4c00066

[2] Weiss et al., Guided Diffusion for Inverse Molecular Design, Nature Computational Science (2023). https://doi.org/10.1038/s43588-023-00532-0

[3] Schneuing et al., Structure-Based Drug Design with Equivariant Diffusion Models, Nature Computational Science (2024). https://doi.org/10.1038/s43588-024-00737-x

[4] Sample Efficient Reinforcement Learning with Active Learning for Molecular Design, Chemical Science (2024). https://doi.org/10.1039/D3SC04653B

[5] Directly Optimizing for Synthesizability in Generative Molecular Design Using Retrosynthesis Models, Chemical Science (2025). https://doi.org/10.1039/D5SC01476J

[6] PropMolFlow: Property-Guided Molecule Generation with Geometry-Complete Flow Matching, Nature Computational Science (2026). https://doi.org/10.1038/s43588-025-00946-y

[7] Chemistry-Inspired Diffusion with Non-Differentiable Guidance, ICLR (2025). https://proceedings.iclr.cc/paper_files/paper/2025/hash/1ba3a079c5922a70929316fda4389eb8-Abstract-Conference.html

[8] Guevorguian et al., Small Molecule Optimization with Large Language Models (2024). https://arxiv.org/abs/2407.18897

[9] Closed-Loop Molecular Inverse Design with Semantic LLM Surrogates (2026). https://arxiv.org/abs/2608.22967

[10] What Are LLM Agents Actually Learning on Low-Budget Molecular Optimization Benchmarks? (2026). https://arxiv.org/abs/2611.08522

[11] Saturn: Sample-Efficient Generative Molecular Design Using Memory Manipulation (2024). https://arxiv.org/abs/2405.17066

The benchmark is dominated by lead optimization, making the overall success rate misleading. The authors could report a macro-averaged score across task categories as a primary metric.

Here is a table with the averaged score across task categories:

ModelOverall success (%)Averaged success across categories (%)
Claude Sonnet 4.638.017.1
GPT-5.440.215.0
Gemini 3.1 Pro39.016.3
Kimi K2.5 Thinking30.311.5
DeepSeek V3.224.39.3
MiniMax M2.719.39.0
Qwen3.5 397B A17B27.59.2

To what extent do the observed failures reflect limitations of the underlying LLMs versus limitations of the agent harness? Would stronger planning, memory, or candidate-selection mechanisms improve performance?

Please see our response above! We believe that the primary modes of failure (planning and predicting the effects of multi-step molecular edits) reflect limitations within the underlying base model as opposed to fundamental limitations with the LLM harness itself. The mechanism through which a hyper-specialized LLM harness can provide fruitful gains in performance is unclear from reading the raw rollouts.

Could the authors clarify whether domain experts were involved in designing or validating the benchmark tasks? Assuming that all tasks are practically important for small molecule drug design, is it common that they are solved using LLM agents rather than specific software?

Please see our response above where we discuss how each of the tasks were extensively validated by domain experts. To our knowledge, the tasks in SMDD-Bench are predominantly typically solved by human experts instead of specific software. This fact combined with the overall importance of the field of small molecule drug design leads us to create this benchmark for LLM agents.

How robust are the reported results across independent runs? Could the authors report mean performance and standard deviations (or confidence intervals) over multiple runs?

Please see our response above where we show variance metrics across multiple independent runs.

AuthorsOpenReview ↗

Author Response 3

How confident can we be that improvements in benchmark performance correspond to meaningful improvements in real-world molecular properties rather than optimization of the surrogate models like Boltz-2 or ADMET AI?

Ultimately, small molecule drug design is a hard problem in large part due to how slow wet lab verification is. Recently, computational drug design as a field has converged on accepting computational surrogate models as oracles of sufficient fidelity. Please see the list above for a list of both seminal and recent works following this same perspective.

Could the authors additionally report a macro-averaged success rate across all five task types?

Please see our response above.

In conclusion, we’re happy to have been able to respond to each and every one of your concerns. If you have any additional concerns, please do let us know!

Reviewer_Y43xOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SMDD-Bench, a benchmark comprising 502 instances across five small-molecule drug design tasks. Each task is constructed around a hidden solution that satisfies its constraints, ensuring feasibility. LLMs perform multiple rounds of design and validation under a limited tool-call budget. The experiments compare seven models and analyze candidate selection, output diversity, and planning across rounds.

Critical strengths

The tasks are practically relevant. The benchmark covers pharmacophore identification, hotspot discovery, scaffold hopping, lead optimization, and fragment assembly, evaluating iterative design and tool-use capabilities.

The task construction procedure is clear. By first identifying a valid solution and then constructing the task and its constraints, the benchmark avoids evaluating models on infeasible problems.

The paper provides valuable process-level analyses. Beyond success rates, it examines the gap between candidate generation and selection, repetitive outputs, and failures in planning across rounds, helping pinpoint weaknesses in the agents.

Critical weaknesses

The scoring provides limited differentiation on difficult tasks. Success rates for hotspot discovery, scaffold hopping, and fragment assembly are generally close to zero, while the requirement to satisfy all constraints treats near-successes and complete failures alike. The benchmark should report stage-specific scores, constraint satisfaction rates, and geometric errors to better distinguish model capabilities and guide improvements.

The model baselines need updating. The evaluation should prioritize adding recent representative models, such as Claude Opus 5.5 and GPT-6. The existing results focus on earlier models within a simplified ReAct framework and are insufficient to characterize the current drug design capabilities of LLM agents. Some task-specific baselines are also weak; for example, hotspot discovery is compared only against random sampling.

The study lacks controlled comparisons of how 3D information is presented. Providing raw PDB/SDF files and allowing code-based analysis does not necessarily mean that models can effectively extract key spatial relationships. The authors should compare raw coordinates, structured descriptions of distances and interactions, and multi-view structural images to distinguish barriers in input representation and tool use from genuine limitations in 3D reasoning.

There is a gap between computational scores and real-world drug design capabilities. The design tasks are primarily scored using Boltz2 and ADMET-AI, and the agents use these same tools for optimization. Passing the benchmark therefore directly demonstrates the ability to meet predictor-defined criteria; it should not be equated with experimentally validated binding, ADMET performance, or successful experimental design.

Questions for authors

see the weaknesses

2 comments
AuthorsOpenReview ↗

Author Response 1

We would like to thank the reviewer for the review. We are pleased to hear that you enjoyed the practical relevance and analyses of our work. Here are our point-by-point responses to your concerns.

The scoring provides limited differentiation on difficult tasks. Success rates for hotspot discovery, scaffold hopping, and fragment assembly are generally close to zero, while the requirement to satisfy all constraints treats near-successes and complete failures alike. The benchmark should report stage-specific scores, constraint satisfaction rates, and geometric errors to better distinguish model capabilities and guide improvements.

SMDD-Bench is a benchmark for real-world small molecule design tasks. Our goal is simply to benchmark the current capabilities of various LLMs regardless if the LLMs perform similarly or differentiate themselves. With that being said, we did take a look into which metrics from the benchmark that can be used as “partial progress”:

ModelInteraction Point Discovery error (Å)Scaffold Hopping interaction sim.Fragment Assembly pose RMSD (Å)
GPT-5.44.090.533.53
Gemini 3.1 Pro3.760.503.06
Claude 4.6 Sonnet4.680.483.74
Kimi K2.5 Thinking4.820.464.42
Qwen3.5 397B-A17B4.630.404.68
DeepSeek V3.25.130.443.77
MiniMax M2.74.840.353.89

Table 12 in the appendix also already shows the number of partially correct interaction point discovery tasks solved by each model.

The model baselines need updating. The evaluation should prioritize adding recent representative models, such as Claude Opus 5.5 and GPT-6. The existing results focus on earlier models within a simplified ReAct framework and are insufficient to characterize the current drug design capabilities of LLM agents. Some task-specific baselines are also weak; for example, hotspot discovery is compared only against random sampling.

The agents were benchmarked using a hand-built and optimized LLM harness. The ReAct framework referenced in Appendix C was from an earlier version of the manuscript. We will remove this sentence and elaborate on the agentic harnesses we used in the manuscript.

Evaluating Claude Opus 5.5 led to 0% in all tasks due to bio-safety refusals from Anthropic. To still get a sense of the performance from the latest closed source frontier models, we evaluated Opus 5 and GPT 6 sol. The results are in the following table:

Task typeOpus 5GPT 6 Sol
2D Pharmacophore Identification12.0%20.0%
Interaction Point Discovery20.0%16.0%
Scaffold Hopping1.9%17.3%
Lead Optimization71.5%62.9%
Fragment Assembly5.0%1.7%
Overall success rate50.8%46.4%

Random sampling is the most effective baseline to determine if the LLM is truly understanding protein-ligand interactions as opposed to getting lucky with random guessing.

The study lacks controlled comparisons of how 3D information is presented. Providing raw PDB/SDF files and allowing code-based analysis does not necessarily mean that models can effectively extract key spatial relationships. The authors should compare raw coordinates, structured descriptions of distances and interactions, and multi-view structural images to distinguish barriers in input representation and tool use from genuine limitations in 3D reasoning.

Because of the agentic setup of the benchmark, the LLM agent is free to decide how it performs the 3D analysis of the key spatial relationships. Our goal is to provide the benchmark in order to a) develop a standardized evaluation method for SMDD agents and understand current capabilities and limitations and b) generate excitement about LLM agents for drug discovery while simultaneously lowering the barrier to entry for ML scientists to contribute to AI for science. We believe the harness optimizations and techniques you mention are out of scope for the current work and are more suited for those training LLMs or developing agents.

AuthorsOpenReview ↗

Author Response 2

There is a gap between computational scores and real-world drug design capabilities. The design tasks are primarily scored using Boltz2 and ADMET-AI, and the agents use these same tools for optimization. Passing the benchmark therefore directly demonstrates the ability to meet predictor-defined criteria; it should not be equated with experimentally validated binding, ADMET performance, or successful experimental design.

The way SMDD-Bench relies on strong computational surrogates such as Boltz2 and ADMET-AI is in line with the accepted norms of the field of computational small molecule drug design [1-11]. Due to the intractability of the wet lab validating each result, computational surrogates have become an accepted oracle. Nowhere in the paper do we make claims regarding experimentally validated binding, ADMET performance, or successful experimental design of wet lab procedures.

[1] Guo and Schwaller, Augmented Memory: Sample-Efficient Generative Molecular Design with Reinforcement Learning, JACS Au (2024). https://doi.org/10.1021/jacsau.4c00066

[2] Weiss et al., Guided Diffusion for Inverse Molecular Design, Nature Computational Science (2023). https://doi.org/10.1038/s43588-023-00532-0

[3] Schneuing et al., Structure-Based Drug Design with Equivariant Diffusion Models, Nature Computational Science (2024). https://doi.org/10.1038/s43588-024-00737-x

[4] Sample Efficient Reinforcement Learning with Active Learning for Molecular Design, Chemical Science (2024). https://doi.org/10.1039/D3SC04653B

[5] Directly Optimizing for Synthesizability in Generative Molecular Design Using Retrosynthesis Models, Chemical Science (2025). https://doi.org/10.1039/D5SC01476J

[6] PropMolFlow: Property-Guided Molecule Generation with Geometry-Complete Flow Matching, Nature Computational Science (2026). https://doi.org/10.1038/s43588-025-00946-y

[7] Chemistry-Inspired Diffusion with Non-Differentiable Guidance, ICLR (2025). https://proceedings.iclr.cc/paper_files/paper/2025/hash/1ba3a079c5922a70929316fda4389eb8-Abstract-Conference.html

[8] Guevorguian et al., Small Molecule Optimization with Large Language Models (2024). https://arxiv.org/abs/2407.18897

[9] Closed-Loop Molecular Inverse Design with Semantic LLM Surrogates (2026). https://arxiv.org/abs/2608.22967

[10] What Are LLM Agents Actually Learning on Low-Budget Molecular Optimization Benchmarks? (2026). https://arxiv.org/abs/2611.08522

[11] Saturn: Sample-Efficient Generative Molecular Design Using Memory Manipulation (2024). https://arxiv.org/abs/2405.17066

Once again, we would like to thank the reviewer for the review and hope we were able to address each and every one of your concerns!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.