- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes MEDD, an LLM-agent framework for structure-based molecular optimization using multi-pose docking evidence. The method aggregates protein–ligand interactions across multiple docking poses to distinguish stable contacts from variable optimization opportunities. An LLM agent then generates interaction hypotheses and constrained molecular modifications, followed by redocking and structural verification. Experiments on CrossDocked2020 demonstrate improvements over the initial MolCRAFT-generated molecules in docking scores, drug-likeness, synthetic accessibility, and molecular reasonability. The authors further provide component ablations and structural case studies to support the proposed framework.
Critical strengths
1. Well-motivated structural evidence integration.
The paper addresses a meaningful limitation of docking-guided molecular optimization: relying on a single docking pose may introduce unreliable structural guidance. Aggregating interactions across multiple poses is a reasonable strategy for reducing sensitivity to individual predicted configurations.
2. Interpretable hypothesis-driven optimization.
The explicit hypothesis–editing–verification workflow is conceptually appealing. Separating LLM reasoning from chemical transformations and structural evaluation improves the traceability of molecular modifications and provides a clearer interpretation of optimization decisions.
3. Comprehensive evaluation of molecular properties.
The experiments consider multiple complementary metrics, including Vina scores, QED, SA, MRR, and structural similarity. The ablation studies also suggest that multi-pose evidence, explicit hypotheses, and verification feedback contribute to optimization performance.
Critical weaknesses
W1. Limited methodological novelty beyond existing molecular optimization frameworks.
The main contribution primarily involves integrating multi-pose interaction fingerprints, LLM-based molecular editing, and docking-based verification into an iterative optimization workflow. These components are individually well established, while the proposed cross-pose consistency measure is essentially an interaction occurrence frequency. The paper does not sufficiently demonstrate a substantial methodological advance beyond their integration.
W2. Questionable assumptions underlying multi-pose evidence and potentially over-constrained molecular optimization.
The method assumes that interactions consistently observed across docking poses provide more reliable structural evidence. However, poses generated by the same docking algorithm may be highly correlated, and interaction frequency does not necessarily reflect physical interaction stability. More importantly, preserving stable contacts or their associated molecular motifs may unnecessarily restrict the chemical search space, since alternative functional groups or binding geometries can establish equally favorable or stronger interactions with the same protein residues. This potential exploration–exploitation trade-off is not adequately investigated and may partly explain MEDD's inferior performance relative to CIDD.
W3. Lack of sensitivity analysis for core design choices.
The number of docking poses, evidence thresholds ( and ), and refinement rounds are central hyperparameters that directly affect structural evidence construction and optimization behavior. The authors explicitly acknowledge that controlled sensitivity analyses were not performed. Without evaluating these choices, it is difficult to determine whether the reported improvements are robust or depend on specific experimental configurations.
W4. Insufficient quantitative validation of the structural verification mechanism.
A central claim is that MEDD verifies whether hypothesized interaction changes are realized while preserving important contacts. However, the experimental evaluation primarily emphasizes conventional molecular quality metrics and qualitative structural examples, without sufficiently detailed quantitative evidence on hypothesis verification success and interaction preservation. Furthermore, verification relies on the same docking-based framework used to generate the initial evidence, raising questions about whether it provides independent evidence of improved binding interactions.
W5. Missing computational efficiency analysis and budget-matched comparisons.
MEDD requires multiple docking poses, repeated structural analysis, and several rounds of LLM reasoning and molecular editing, potentially introducing substantial computational overhead. However, the paper does not report wall-clock runtime, LLM token consumption, the number of docking evaluations, or overall computational cost. Comparisons with CIDD and simpler optimization baselines under matched computational budgets would be important for assessing whether the additional complexity is justified, especially given that MEDD underperforms CIDD on the main endpoint metrics.
Questions for authors
-
Structural evidence and search-space constraints: How do the authors account for correlations among docking poses when estimating interaction consistency? Furthermore, can the framework explore alternative functional groups or binding geometries that preserve or improve favorable interactions without retaining the original molecular motifs? Could the current preservation strategy limit optimization performance?
-
Hyperparameter sensitivity: How sensitive is MEDD to the number of docking poses, the stable/variable interaction thresholds, and the refinement budget? Can the authors provide sensitivity analyses for these key design choices?
-
Structural verification: Can the authors provide quantitative results on hypothesis verification success rates and key-interaction preservation rates? Is there any independent structural assessment beyond the docking protocol used to construct the original evidence?
-
Computational efficiency and comparison fairness: What are the average runtime, LLM token consumption, and number of docking evaluations per optimization task? How does MEDD compare with CIDD and simpler optimization baselines under matched computational budgets?
5 comments
Response to Q2: Hyperparameter Sensitivity
We thank the reviewer for this suggestion. Our configuration was chosen to maintain comparability with CIDD in LLM backbone and nominal candidate budget while balancing structural evidence, refinement quality, and cost. We further conducted 42 actual MEDD runs on six fixed tasks across seven configurations, varying one parameter at a time.
Hyperparameter definitions. MEDD contains three key hyperparameters:
1) Number of docking poses (K): The maximum number of Vina-generated binding poses used to construct multi-pose interaction evidence. A larger K provides more configurations for identifying recurring and pose-dependent interactions.
2) Refinement budget (M): The maximum number of candidate-generation attempts during hypothesis-guided refinement. It controls editing opportunities and associated LLM/docking costs.
3) Stable-contact threshold (τ): The minimum fraction of valid docking poses supporting an interaction for it to be classified as stable. The formal setting τ=1.0 requires an interaction to appear in every valid pose, while lower thresholds classify more contacts as stable, potentially strengthening preservation guidance.
The formal configuration is K=20, M=5, and τ=1.0. We evaluated K∈{5,10,20}, M∈{3,5,7}, and τ∈{0.5,0.75,1.0}, using the same initial molecules, protein pockets, and LLM model identifier across configurations.
Sensitivity analysis results
| Configuration | Mean ΔVina ↓ | Success ↑ | Contact Retention ↑ | LLM Calls | Docking Calls | Time (s) |
|---|---|---|---|---|---|---|
| K=5 | −0.754 | 50.0% | 0.802 | 37.0 | 6.0 | 380.6 |
| K=10 | −1.610 | 33.3% | 0.835 | 36.4 | 6.0 | 404.9 |
| K=20 (default) | −1.770 | 50.0% | 0.890 | 37.4 | 5.8 | 360.1 |
| M=3 | −1.261 | 33.3% | 0.835 | 23.0 | 4.0 | 262.8 |
| M=7 | −1.359 | 16.7% | 0.820 | 52.2 | 7.4 | 437.1 |
| τ=0.5 | −1.499 | 66.7% | 0.751 | 38.2 | 5.6 | 351.7 |
| τ=0.75 | −1.273 | 50.0% | 0.884 | 37.4 | 5.8 | 358.8 |
Mean ΔVina, Contact Retention, and resource consumption are calculated over completed tasks, while Success is evaluated over all six predefined tasks per configuration.
1. Docking pose count (K). Increasing K from 5 to 20 improves mean ΔVina from −0.754 to −1.770 kcal/mol and Contact Retention from 0.802 to 0.890, while maintaining comparable LLM usage. This suggests that additional pose evidence may support refinement. Although Success is not monotonic, K=20 gives the strongest combined affinity improvement and retention on this small task subset, without more candidate-generation calls.
2. Refinement budget (M). Reducing M from 5 to 3 decreases average LLM calls from 37.4 to 23.0 and runtime from 360.1 to 262.8 seconds, but also reduces Success from 50.0% to 33.3% and increases the risk of failing the minimum valid-candidate requirement. Conversely, increasing M to 7 substantially increases LLM calls (52.2) and runtime (437.1 seconds), without improving conventional Success (16.7%). The additional cost of M=7 does not yield consistent gains across metrics. Thus, M=5 offers a practical balance while matching CIDD's nominal attempt budget.
3. Stable-contact threshold (τ). The default τ=1.0 emphasizes interactions consistently observed across all valid poses. Relaxing τ to 0.5 increases Success from 50.0% to 66.7%, but reduces Contact Retention from 0.890 to 0.751 and yields a less favorable mean ΔVina (−1.499 versus −1.770 kcal/mol). At τ=0.75, Contact Retention remains relatively high (0.884), without improving other primary outcomes. These results indicate a trade-off in selecting stable-contact evidence: lowering τ labels more contacts as stable, but does not necessarily improve their final retention. Given the six-task sample, the trend should be interpreted cautiously.
Overall, K=20, M=5, and τ=1.0 reflect a resource-aware, comparability-oriented design rather than post-hoc selection for one metric. These small-scale sensitivity results are consistent with the intended balance among evidence quality, refinement performance, and cost.
Detailed results and configurations are provided in the supplementary materials.
Response to Q3: Structural Verification
We thank the reviewer for this question. We conducted a retrospective analysis of formal MEDD records to quantify hypothesis realization, key-interaction retention, and post-hoc structural consistency, without modifying the original optimization procedure.
1. Quantitative hypothesis verification.
We introduce the Hypothesis Realization Rate (HRR) to quantify the proportion of proposed structural hypotheses supported by the final selected molecule's interaction evidence. Unlike endpoint Success, HRR assesses whether proposed interaction changes appear in the final predicted structures.
Specifically, MEDD considers two types of hypotheses:
1) Stabilize variable interactions: A hypothesis is verified when the targeted interaction exhibits higher occupancy across valid final docking poses than in the initial multi-pose evidence.
2) Preserve stable interactions: A hypothesis is verified when the targeted interaction remains present across all valid final poses.
Interactions are matched by interaction type and protein chain/residue identity. Importantly, improved Vina scores alone do not constitute successful hypothesis realization.
HRR is calculated as:
From 19,948 hypotheses across 836 formal tasks, we obtained:
| Outcome | Count |
|---|---|
| Verified | 6,695 |
| Not Verified | 13,224 |
| Not Assessable | 29 |
| HRR | 33.61% |
| 95% CI | [32.43%, 34.87%] |
The 95% CI uses 10,000 pocket-clustered bootstrap resamples. The 29 non-assessable hypotheses were excluded because final structural evidence was missing or invalid.
Additionally, 788 of 834 assessable tasks (94.48%) contained at least one verified hypothesis. This indicates that some proposed interaction objectives are realized in final predicted structures. HRR is not experimental confirmation or a per-iteration verification success rate.
2. Key-interaction preservation.
We further quantified Stable Interaction Retention (SIR), defined as the proportion of initially identified Stable/Protected interactions retained across all valid final poses.
The formal records contained 141 stable interactions across 18 tasks, of which 92 were preserved, yielding a strict SIR of 65.25% (92/141), with a pocket-clustered 95% CI of [42.64%, 100.00%]. Under a less restrictive criterion requiring preservation in at least one final pose, retention increased to 87.94% (124/141).
We report the strict criterion as primary to match MEDD's definition. Since qualifying contacts occur in only 18 tasks, SIR has limited coverage and a wide CI. CIDD SIR cannot be recovered comparably from its historical records.
To provide a broader comparison, we also performed a unified PLIP 3.0.1 evaluation on 820 paired MEDD–CIDD tasks:
| Metric | CIDD | MEDD | Paired difference (95% CI) |
|---|---|---|---|
| Contact Retention ↑ | 46.18% | 52.65% | +6.47 pp [4.30, 8.64] |
| PLIF Retention ↑ | 42.36% | 48.98% | +6.61 pp [4.60, 8.64] |
These results favor MEDD in predicted interaction retention under a common protocol. Unlike SIR, these measures cover broader contact patterns in both methods.
3. Independent structural assessment.
We separately re-evaluated persisted MEDD and CIDD structures using a unified PLIP 3.0.1 protocol, without relying on Agent decisions or Vina scores to judge interaction preservation.
However, these structures retain coordinates from the original docking workflow. This is a separate post-hoc interaction assessment, not validation by an independent docking engine, molecular dynamics, or experimental structures.
Overall, the retrospective evidence gives HRR = 33.61% and strict SIR = 65.25%; the 820-task PLIP comparison also favors MEDD in Contact and PLIF Retention. These results support hypothesis-guided refinement at the level of predicted structures, without establishing physical binding mechanisms. Full definitions and statistics are included in the supplementary materials.
Response to Q1: Structural Evidence and Search-Space Constraints
We thank the reviewer for these insightful questions. We clarify the motivation behind multi-pose evidence, the flexibility of our editing strategy, and its associated optimization trade-offs.
1. Multi-pose evidence and pose correlations.
The motivation for multi-pose analysis is to mitigate uncertainty in predicted binding orientations and potential pose variations, rather than relying on a single docking configuration. In our implementation, Vina generates up to 20 poses, and PLIP extracts their protein–ligand interactions. Contacts observed in every valid pose (support_count == valid_pose_count) are classified as stable, while partially supported contacts are treated as variable. These observations are aggregated into structural hypotheses and passed to DesignAgent as editing guidance.
We acknowledge that docking poses are correlated. Therefore, cross-pose consistency serves as heuristic structural evidence, not a calibrated stability probability or a collection of independent observations. Explicit pose clustering and correlation-aware weighting are not implemented in the current framework, and we do not claim otherwise.
2. Flexible exploration through hypothesis-guided editing.
MEDD does not assume that LLM agents possess stronger 3D structural perception than pocket-conditioned deep generative models. Instead, the initial generator provides a structure-informed molecule, while the agents incorporate chemical knowledge, interaction evidence, and iterative feedback to guide refinement.
Specifically, DesignAgent proposes chemical modifications based on stable/variable interactions and protected-region guidance; MolGenAgent generates candidate SMILES; and RDKit, Vina, and PLIP provide validity and structural evaluation. ReflectionAgent incorporates the resulting feedback, and ChooseAgent selects the final candidate.
Controlled editing is intended to make structural hypotheses traceable and testable, not to freeze the original molecular motifs. Preserving favorable interactions does not necessarily require retaining the same functional groups. The implementation allows scaffold and substituent modifications outside explicitly protected subgraphs, while contact loss generally informs subsequent decisions rather than serving as an unconditional rejection rule.
This flexibility is reflected in our supplementary results: 93.9% (921/981) of MEDD outputs underwent chemical structural changes, and only 13.46% retained identical Murcko scaffolds. Among 770 paired tasks where both methods performed chemical edits, MEDD achieved higher Contact Retention (+6.31 percentage points) and PLIF Retention (+6.42 percentage points). These results support structural flexibility alongside interaction preservation, although they do not directly establish the success of individual functional-group substitutions.
3. Preservation–optimization trade-off.
We agree that structural constraints can restrict exploration, particularly when substantially different binding geometries are preferable. Indeed, CIDD achieves greater unconstrained mean Vina improvement than MEDD (−0.8083 versus −0.6500 kcal/mol, 981 paired tasks).
However, MEDD targets evidence-guided refinement that balances molecular improvement with interaction preservation. Our supplementary evaluation shows:
Contact Retention, PLIF Retention, and SCIR are evaluated on 820 paired tasks with recoverable poses; SPIR and SPJIR use 981 paired tasks. These results demonstrate stronger structure-constrained improvement, rather than universal superiority in unconstrained optimization. We acknowledge that the specific effect of relaxing preservation constraints requires further controlled analysis.
Implementation and reproducibility.
To clarify these mechanisms, we provide the implementation details, relevant code locations, complete structured Agent prompt templates, and their dynamic input/output specifications in the supplementary materials. These cover multi-pose interaction aggregation, structural-hypothesis construction, protected-edit constraints, candidate generation, structural verification, and final selection.
Overall, MEDD uses multi-pose evidence to support chemically informed Agent decisions and hypothesis-guided refinement, while allowing molecular modifications and explicitly acknowledging the trade-off between preservation and unconstrained exploration.