- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces UMO-Agent, an LLM-based agentic framework for fragment-based molecule optimzation. UMO works by editing the initial molecule iteratively. In each iteration, a challenger molecule is selected and compared with the currently retained one, from different aspects. Compared to existing works of agentic molecule optimization, the key claimed technical novelty of UMO is that it takes uncertainty and risks within the editing and decision steps into consideration. On the CrossDocked benchmark, UMO achieves a higher success rate than the reported baselines. Ablation studies support the effectiveness of different components in UMO.
Critical strengths
-
Considering uncertainty and risks in the decision process of molecule optimization is meaningful and well-motivated.
-
Perfoming local optimization and then full docking can effectively reduce cost.
-
Reported experimental results are good. Ablation studies clearly show the performance gains from different components.
Critical weaknesses
-
As highlighted in the title, abstract and introduction section, considering uncertainty is the core technical contribution of this paper. However, as can be seen from the experimental results, even UMO w\o MCDropout outperforms baselines like CIDD significantly. This indicates that the good performance of UMO comes from the workflow as a whole, in which the contribution of other components also plays an important role.
-
A higher SR alone cannot explain how and when the uncertainty-based component works, particularly given that uncertainty signal does not trigger the critical "RED" flag on its own. The authors should design and conduct experiments to investigate how this components works, for example, how uncertainty signals alter the decision of the critic and the controller.
-
Some important details are missing: the precise definitions and computation of several evidence-vector components, the calculation of "edit radius", the rules for building the library, etc.
-
MCDropout requires access to a predictor with dropout enabled at inference time. How can UMO accommodate a strong black-box predictor for which MC dropout is unavailable? Additionally, five forward passes for MCDrop may be not enough; please provide an sensitivity analysis on the number of forward passes. Furthermore, it is not clear whether the uncertainty quantified by MCDrop correlates well with prediction errors; please provide evidence for this.
Questions for authors
Please see "weaknesses". One more: why is the Boltzmann distribution chosen? How sensitive is the perforance to the choice of the distribution family here?
Response to Reviewer qqkz
Thanks for your constructive comments and questions. Below we address Weaknesses 1–4 and Question 1.
Removing only MC-Dropout reduces Joint SR from 60% to 50% and increases mean ΔHA from +2.2 to +6.6 (Table 2). Table G4 further isolates UQ with and without LLM guidance: enabling UQ improves SR from 55% to 56% without the LLM, and from 50% to 60% with the LLM, while reducing mean HA from 29.2 to 24.8 in the latter setting. These results suggest that UQ is particularly useful in the LLM-guided workflow.
In Figure 1 and Eq. (4), ut enters et as an auxiliary signal for the
Evidence Critic(Ctevid) andCtrl. It can affect risk assessment for borderline proposals and corroborate other anomalies, but does not triggerREDon its own. In our 100-target audit (General Response Table G3c), enabling ut increasesGREEN-conditional acceptance (85.1% -> 94.9%), reduces verified steps (428 -> 341) and committed edits (236 -> 191), and limits molecular growth (ΔHA:+6.6→+2.2). Because these are different stochastic trajectories, we do not interpret the aggregate differences as matched-state decision changes.Thanks for pointing this out. We will detail all three in Appendix A.3–A.7:
{GREEN, YELLOW, RED}.LOCAL≤0.52,BROAD≤0.75).[1](5,563 fragments), we retain 1,164 fragments that pass RDKit validity checks and contain at most six heavy atoms (≤6 HA).Black-box compatibility. MC-Dropout can be replaced by other uncertainty estimates when available. With point predictions only, UMO-Agent retains two-tier verification, evaluator disagreement, and memory. However, removing UQ reduces Joint SR from 60% to 50% and increases molecular growth (+2.2 to +6.6 HA), highlighting its contribution.
Forward-pass sensitivity. Using the same populations as Table G3a, varying T from 3 to 30 yields AUROC of 0.876–0.908 (Table G3b). Five passes (0.883 AUROC) provide a practical balance between discrimination and cost.
Relationship with prediction errors. Lead optimization requires correctly ranking similar ligands by affinity to determine whether an edit improves upon the current ligand. In Section 4.4, even FEP+ achieves a median Spearman correlation of only 0.62 on congeneric series (docking: 0.32; NN: 0.24). Thus, we combine UQ with LLM-guided decisions to mitigate risk, even at the cost of rejecting potentially beneficial edits when evidence is insufficient. Uncertainty remains relatively stable on real analogues but rises on potentially reward-hacked states even for small heavy-atom changes (Table G3a). Adding UQ to the LLM-guided workflow improves SR from 50% to 60% and reduces molecular growth (Tables G3c–G4). This supports UQ as a complementary risk signal, not a direct detector of ranking errors.
We use an ϵ-mixed Boltzmann distribution to convert combined proposal scores into stochastic action preferences, with temperature (T=0.70), uniform exploration (ϵ=0.20), and probability capping (0.70) to balance exploration and exploitation (60%±3% SR across three seeds). Replacing this score-guided Boltzmann distribution with uniform sampling over edit factors while keeping verification, UQ, and risk control intact (
w/o Decision, Table 2) reduces Joint SR to 40%±2%.We appreciate your constructive feedback and hope these responses help address your questions.
References
[1]Green H, Koes DR, Durrant JD. DeepFrag: a deep convolutional neural network for fragment-based lead optimization. Chem. Sci., 2021.