acceptodds
Under review as a conference paper at ICLR 2027

Pharmacological Mechanism Reasoning with Collaborative LLM Agents for Molecular Property Prediction

Abstract

Molecular property prediction has advanced on the premise that properties follow from a sufficiently precise representation of structure. LLMs have extended this paradigm by aligning molecular and textual representations, retrieving analogous molecules, and learning step-wise reasoning over structure. In all of these, the property is still predicted directly from structure, without passing through the biological context in which it arises. A molecular property, however, is observed as the outcome of a molecule engaging targets in vivo and triggering downstream events, and a prediction that does not pass through that mechanism remains an extrapolation of structural similarity. We reformulate property prediction as mechanism-grounded evidence reasoning and propose PharmaLens, a multi-agent framework that realizes this formulation. For a given property, agents at the physicochemical, target-relation, and binding-structure levels establish the specified factors on the query molecule by tool-based measurement, and the resulting evidence is integrated over the relational logic. Across in vivo endpoints from TDC and organ-toxicity tasks from UniTox, PharmaLens achieves strong performance on both benchmarks, and it retains this margin on property cliffs, where structurally similar molecules carry opposite labels and label-trained predictors fall off most sharply. Its rationales are also consistently favored across all three evaluations. The source code is publicly available at https://github.com/anonymousqwerty350/PharmLens.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
LLM agents, drug discovery, molecular property prediction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8rLzOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper introduces PharmLens, which uses multiple agents to obtain literature-based evidence and applies it to the prediction of molecular properties. PharmLens collects ligand knowledge using physicochemical agents, and pathway knowledge related to targets using protein-relevant and interaction agents; if lacking, it additionally finds reference ligands that bind to the relevant targets from the document, verifies their specific binding, and then compares them with the query molecule. The evidence obtained in the above process (LCE, PCE, IE) is combined to predict the molecular characteristics. Experiments were conducted using TDC and UniTox dataset, and additionally, a property-cliff distinction experiment was performed, showing overall superior performance compared to several existing llms.

Critical strengths

For predicting molecular properties, It is a good approach to use multiples agents to collect various pieces of evidence and determine the property by considering a robust methodlogy, especially using pathway information.

In both evaluation datasets(TDC & UniTox), this methodology contributed to the highest performance and score improvements among multiple LLMs. In particular, it demonstrated higher discriminative ability than other models when evaluating the cliff subset which is hard to distinguish. Moreover, after conducting Rationale evaluations with an LLM judge, human evaluators and fine-tuning ModernBERT, it demonstrated acceptable quality of reasoning.

Critical weaknesses
  • Several choices in the docking pipeline are not sufficiently justified. First, the criteria for selecting a PDB structure when multiple structures are available for a protein are unclear. Second, only the top-ranked docking pose is used for PLIP analysis, without examining the effects of alternative poses. Finally, restricting docking to a single binding site may overlook relevant interactions at alternative or allosteric sites. These choices raise concerns about the reliability of the resulting interaction evidence.

  • The experimental evaluation has two limitations. First, comparisons with more recent frontier LLMs, such as GPT-5.x and newer Claude models, are missing, making it difficult to assess the framework's effectiveness against stronger contemporary baselines. Second, the UniTox evaluation aggregates eight toxicity endpoints into four physiological-system groups and reports group-level averages rather than individual endpoint results. This aggregation may obscure substantial performance variations across endpoints and potentially overstate the consistency of the reported improvements.

  • The ablation results show that PCE and IE provide relatively small improvements, with all Macro-F1 gains below 0.1. This raises questions about the effectiveness of these components given the added complexity. In addition, the paper does not include an ablation study for LCE, making it difficult to evaluate its contribution.

  • Although the paper evaluates rationale quality using LLM judges and human experts, it lacks detailed case studies showing how intermediate evidence leads to final predictions. In particular, the absence of failure case analysis makes it difficult to assess the reliability of the proposed mechanistic reasoning process.

  • The manuscript contains several inconsistencies and typos:

    • Model naming: The framework is inconsistently referred to as PharmLens, PharmaLens, and TheKnow.
    • Table reference: On Line 304, "(backbone ablation table)" should be replaced with "Table 3".
    • Model categorization: TxGemma is categorized under "Domain LLMs" in Tables 1 and 2, but under "Multi-Agents" in Table 4.
    • Typographical error: Table 5 contains "Qwnen" instead of "Qwen".

    These issues should be corrected to improve the manuscript's clarity and consistency.

Questions for authors

Q1. Experimental Evaluation

Could the authors provide additional results using more recent frontier LLMs, such as GPT-5.x and newer Claude Sonnet models? Additionally, could the authors report the Macro-F1 scores for each of the eight UniTox endpoints separately, rather than only presenting the aggregated results across four physiological-system groups?

Q2. Ablation Study

Could the authors provide an ablation study for LCE to clarify its contribution to the overall performance? Additionally, given the relatively modest absolute improvements from PCE and IE, how do the authors justify the additional complexity of these components?

Q3. Docking Settings

When multiple PDB structures are available for a protein, how is the final structure selected? Are factors such as species, resolution, and protein conformations considered? Additionally, why is only the top-ranked docking pose used for PLIP analysis, and how does the framework handle proteins with multiple binding sites, including allosteric sites?

Q4. Qualitative Analysis

Could the authors provide representative case studies showing the complete reasoning process, including intermediate evidence, protein–ligand relationship predictions, and final property predictions? In particular, failure cases would help assess the reliability of the proposed mechanistic reasoning.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 8rLz (1/4)

We sincerely thank the reviewer for carefully examining our work and raising insightful and thought-provoking questions regarding our methodology. Revisiting our workflow in light of the reviewer's comments allowed us to identify several interesting and meaningful interpretations that we had not fully recognized at the time of submission, ultimately helping us further strengthen our work.

We have organized our responses to the reviewer's comments into the following sections to facilitate the review process.

Q1: Experimental Evaluation and Baseline Comparisons‾\color{red}{\underline{\textsf{Q1: Experimental Evaluation and Baseline Comparisons}}}

[Answer 1-1] Reporting Macro-F1 scores for all eight UniTox endpoints.

[Answer 1-2] Additional evaluations using GPT-5.1 and Claude-Sonnet-5.

Q2: Contributions of Evidence Components and Ablation Analysis‾\color{purple}{\underline{\textsf{Q2: Contributions of Evidence Components and Ablation Analysis}}}

[Answer 2-1] Interpretation of the performance contributions and distinct roles of EPCE\textbf{E}_{\text{PCE}} and EIE\textbf{E}_{\text{IE}}.

[Answer 2-2] Analysis of the ELCE\textbf{E}_{\text{LCE}} ablation results.

Q3: Docking Protocol and Binding-Site Selection‾\color{green}{\underline{\textsf{Q3: Docking Protocol and Binding-Site Selection}}}

[Answer 3-1] Evaluation of alternative and allosteric binding sites.

[Answer 3-2] Top-1/2/3 pose sensitivity analysis.

[Answer 3-3] PDB structure selection criteria and ensemble docking evaluation.

Q4: Qualitative Analysis and Typographical Corrections‾\color{blue}{\underline{\textsf{Q4: Qualitative Analysis and Typographical Corrections}}}

[Answer 4-1] Failure case studies.

Detailed responses to each comment are provided below.


[Answer 1-1] Reporting Macro-F1 scores for all eight UniTox endpoints.‾\color{red}{\underline{\textbf{\textsf{[Answer 1-1] Reporting Macro-F1 scores for all eight UniTox endpoints.}}}}

The original evaluation aggregated the endpoints into four physiological-system groups due to the imbalance in the number of cliff pairs across endpoints. In particular, Ototoxicity contains only six cliff pairs, making its Macro-F1 highly sensitive to changes in a small number of predictions. Therefore, rather than assigning equal weight to performance estimates from endpoints with limited samples, we grouped biologically related endpoints into four systems and reported weighted averages.

Nevertheless, we agree with the reviewer that group-level aggregation may obscure performance variations across individual endpoints. To address this concern, we have added a consolidated table reporting the individual Macro-F1 scores for all eight endpoints, including those of the existing baselines.

Table.1 Performance across eight endpoints. (For clarity, only the top two models in each category are shown; general-purpose LLMs are excluded. Full performance results for all models are provided in the Table.9 of Appendix B.1)

ModelCardioPulmonaryHematoLiverRenalDermatoOtoInfertilityAverageWeighted Mean
General LLMs
Llama3.10.3710.3630.3850.5150.3730.3880.4860.3640.4060.397
GPT-4o0.4530.4410.4780.5170.4880.5480.3330.4100.4590.469
GPT-5.10.4310.4680.3780.5000.4530.5640.6250.4690.4860.463
DeepSeek-V4-Pro0.4560.4640.3650.4820.4560.5440.1430.4870.4250.456
Qwen3.7-max0.4350.4680.3780.5240.4260.5510.3330.5150.4540.465
Claude-Sonnet-50.5190.4680.3890.5460.5090.5920.4860.5240.5040.503
Domain LLMs
Galactica-30B0.4120.5330.3250.3410.3460.5790.3330.3260.3990.393
TxGemma-Chat-27B0.3370.4630.4300.4530.3690.6310.2500.3820.4140.421
Molecular LLMs
LLaMo0.4980.4540.4140.3770.4050.3210.3330.4970.4120.431
EDT-Former0.4910.4890.4960.4880.4400.4490.2500.4810.4480.475
Multi-Agents
CLADD0.4470.4290.3860.4010.6360.4100.3330.4280.4340.446
PharmLens (ours)0.5910.5510.4040.5820.5240.6310.3330.5310.5180.536

At the individual endpoint level, PharmLens achieves the highest performance on five of the eight endpoints. Furthermore, PharmLens obtains the highest performance among all evaluated models in both the unweighted average across the eight endpoints and the weighted average.

Therefore, although performance variations exist across individual endpoints, the overall performance advantage of PharmLens remains consistent across both endpoint-level and physiological-system-level evaluations.

AuthorsOpenReview ↗

Response to Reviewer 8rLz (2/4)

[Answer 1-2] Additional evaluations using recent frontier LLMs‾\color{red}{\underline{\textbf{\textsf{[Answer 1-2] Additional evaluations using recent frontier LLMs}}}}

We thank the reviewer for highlighting the importance of comparisons with more recent frontier LLMs. Accordingly, we evaluated GPT-5.1 and Claude-Sonnet-5 and report their results on the four TDC endpoints and the UniTox cliff set in the Table.2 below.

Table.2 Performance comparison with recent frontier LLMs.

ModelBBBPDILIHIABioavAvg.Circ.Metab.Sens.Repro.Avg.
Llama3.10.4210.4270.5340.5370.4800.3730.4470.4040.3640.397
GPT-4o0.6130.5530.6370.6260.6070.4580.5030.5130.4100.471
GPT-5.10.5940.5960.8010.6630.6630.4240.4780.5740.4690.486
DeepSeek-V4-Pro0.6680.6590.7180.6590.6760.4280.4700.4790.4880.466
Qwen3.7-max0.7010.6150.8050.6160.6840.4260.4770.5160.5150.483
Claude-Sonnet-50.7500.6500.8720.6590.7190.4590.5370.5750.5240.524
PharmLens (ours)0.7260.6690.8750.6940.7410.5160.5540.5830.5310.546

Even against these stronger baselines, PharmLens achieves the highest average Macro-F1 across both evaluations. On TDC, PharmLens yields relative improvements of 11.8% over GPT-5.1 and 3.1% over Claude-Sonnet-5. Similarly, on the UniTox cliff set, it achieves relative improvements of 12.3% and 4.0%, respectively, while outperforming both additional baselines across all four toxicity groups.

These results further demonstrate that the performance advantage of PharmLens is not merely attributable to the choice of comparatively weaker baselines. In particular, this advantage persists on property-cliff molecules, which exhibit high structural similarity despite having opposing property labels. This further supports the utility of acquiring and integrating molecule-specific evidence guided by property-specific mechanistic knowledge, even when compared against stronger frontier LLMs.


[Answer 2] Contributions of Evidence Components and Ablation Analysis‾\color{purple}{\underline{\textbf{\textsf{[Answer 2] Contributions of Evidence Components and Ablation Analysis}}}}

[Answer 2-1] ‾\color{purple}{\underline{\textsf{[Answer 2-1] }}} We understand the reviewer's concern regarding whether the performance gains justify the additional model complexity. However, the practical significance of each component's contribution should be assessed not only in terms of absolute performance gains, but also in the context of task difficulty and performance gaps among competing models.

On the cliff evaluation, the performance drops resulting from removing EPCE\textbf{E}_{\text{PCE}} and EIE\textbf{E}_{\text{IE}} are 5.2 and 2.7 times larger, respectively, than the performance range among the top three baselines (0.017). Moreover, their relative performance improvements increase substantially from 6.1% and 4.4% on the full UniTox dataset to 19.2% and 9.2% on the property-cliff set (Relative improvements were calculated with respect to the corresponding baseline performance). Notably, removing EPCE\textbf{E}_{\text{PCE}} reduces performance below that of the strongest baseline, highlighting its critical role in establishing PharmLens's performance advantage. EIE\textbf{E}_{\text{IE}} further complements EPCE\textbf{E}_{\text{PCE}}'s functional-relation predictions with structural interaction evidence, providing additional performance gains.

[Answer 2-2] ‾\color{purple}{\underline{\textsf{[Answer 2-2] }}} The additional ELCE\textbf{E}_{\text{LCE}} ablation revealed distinct contributions of the evidence layers. (All ablation results are provided in the Appendix C.1.) On the full UniTox evaluation, removing ELCE\textbf{E}_{\text{LCE}} causes the largest performance drop, highlighting its primary contribution to property prediction through intrinsic molecular features. In contrast, removing EPCE\textbf{E}_{\text{PCE}} leads to the largest drop on the cliff set, as shown in the Appendix C.1. These findings suggest that ELCE\textbf{E}_{\text{LCE}} provides a foundation for capturing intrinsic properties, whereas EPCE\textbf{E}_{\text{PCE}} and EIE\textbf{E}_{\text{IE}} contribute to distinguishing molecules with different property outcomes despite structural similarity by assessing their functional relationships and interactions with biological targets. We thank the reviewer for suggesting this additional analysis, which revealed these complementary roles.

AuthorsOpenReview ↗

Response to Reviewer 8rLz (3/4)

[Answer 3-1] Evaluation of alternative binding sites.‾\color{green}{\underline{\textbf{\textsf{[Answer 3-1] Evaluation of alternative binding sites.}}}}

In this study, we prioritized binding sites that are supported by literature and experimental structures as being associated with the target functional relation and offer sufficient reference ligands to construct representative interaction profiles. Accordingly, we did not restrict our evaluation to orthosteric sites or categorically exclude allosteric sites; allosteric sites with established evidence of mediating the same functional relation were also included.

The primary objective of this selection strategy is to construct generalized interaction profiles by integrating interaction patterns from diverse reference ligands sharing the same functional relation. Conversely, indiscriminately including alternative sites with only a limited number of reference ligands may cause the interpretation to rely disproportionately on contact patterns specific to a small number of references.

Nevertheless, we acknowledge the reviewer's concern that this approach may not capture all interactions occurring at other functionally relevant binding sites. To investigate this, we performed additional docking analyses on alternative sites supported by evidence of their association with the same functional relation, constructing separate reference interaction profiles for each site (Appendix A.4.1).

Table.3 Alternative binding-site evaluation results (Macro-F1).

BBBPHIABioavCardioDermatoHematoInfertilityOtotoPulmonaryRenalMean
# Replaced proteins1232313131–
Baseline macro-F10.73020.87540.69390.50540.54010.50580.67000.69860.50870.51040.6238
Δ macro-F1+0.0075−0.0098+0.0165−0.0371−0.0044+0.0233−0.0732+0.0402+0.0251−0.0220−0.0034

The results (Table.3) showed no consistent performance improvements across endpoints, while the overall mean Macro-F1 slightly decreased. This does not imply that alternative sites lack biological significance; rather, it indicates that incorporating alternative-site profiles based on limited reference ligands does not necessarily improve predictive performance or complement the existing representative interaction profile or improve predictive performance.

[Answer 3-2] Top-1/2/3 pose sensitivity analysis.‾\color{green}{\underline{\textbf{\textsf{[Answer 3-2] Top-1/2/3 pose sensitivity analysis.}}}}

In this study, we selected the Top-1 pose, ranked highest by the docking scoring function, as the representative binding hypothesis for each molecule. This choice was not based on an assumption that the Top-1 pose necessarily reflects the experimentally correct binding conformation, but rather on the need to consistently compare the interaction patterns of query and reference ligands using a single predicted binding configuration.

Nevertheless, we recognize that evaluating the sensitivity of predictive performance to pose selection is essential for assessing the robustness of this workflow. Accordingly, we conducted additional sensitivity analyses using the Top-1, Top-2, and Top-3 poses, applying the same selection criteria to both query and reference ligands. As shown below, the average Macro-F1 differences across pose ranks were minimal on both TDC and the UniTox set, with overall predictive performance remaining consistent despite some variations across individual endpoints.

Table.4 Macro-F1 comparison across docking pose ranks on TDC (Top-1, Top-2, and Top-3)

TaskTop-1Top-2Top-3
BBBP0.7260.7300.716
DILI0.6690.6590.671
HIA0.8750.8400.875
Bioavailability0.6940.7180.718
Average0.7420.7370.745
vs. Top-1–−0.005+0.003

Table.5 Macro-F1 comparison across docking pose ranks on UniTox (Top-1, Top-2, and Top-3)

TaskTop-1Top-2Top-3
Cardiotoxicity0.5050.4760.477
Dermatological0.5400.5120.539
Hematological0.5060.5230.505
Infertility0.6700.6520.638
Liver0.6060.6090.620
Ototoxicity0.6990.6990.739
Pulmonary0.5090.5220.524
Renal0.5100.4960.507
Average0.5680.5610.569
vs. Top-1–−0.007+0.001

We thank the reviewer for prompting this sensitivity analysis, which enabled us to explicitly assess the impact of pose selection. The results suggest that average predictive performance remains largely consistent across the evaluated pose ranks. We have incorporated these additional analyses into the revised manuscript.

Reviewer_oNJYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PharmaLens, a multi-agent framework for molecular property prediction guided by pharmacological mechanisms. It first constructs property-specific knowledge from scientific literature, then uses specialized agents to collect physicochemical and protein-relation evidence, with additional docking and contact analysis when needed. An LLM integrates these findings to generate property predictions and explanations. Experiments on TDC and UniTox show improvements over several LLM-based baselines, particularly on property-cliff molecules. The paper also includes ablation studies and evaluations of the generated explanations.

Critical strengths
  1. Using property-specific mechanistic knowledge to guide evidence collection at the physicochemical, protein-relation, and binding-structure levels is a sensible and well-motivated design.
  2. The framework combines multiple specialized tools in a structured workflow, with selective docking and interaction analysis when protein-relation evidence is insufficient.
  3. The experiments cover diverse ADMET and toxicity tasks, showing promising improvements over LLM-based baselines, particularly on property-cliff molecules and across different LLM backbones.
  4. The ablation and mechanism-specification swapping experiments provide useful evidence for the contributions of different components. The additional evaluations of generated rationales, including expert assessments, are also valuable.
Critical weaknesses
  1. The comparisons are largely limited to LLM-based methods, while strong molecular property prediction models such as ChemFM and MolE, as well as conventional supervised ADMET predictors, are missing. Although these methods use different training settings, including them would help better position PharmaLens against established approaches.
  2. It is unclear how much of the improvement comes from mechanism-guided reasoning rather than the additional predictive tools, particularly the supervised QSAR ensembles. Although the paper includes evidence ablations and specification-swapping experiments, a simpler baseline using the same evidence without the proposed reasoning workflow would better isolate its contribution.
  3. Although the protein-relation evaluators are validated through cross-validation, the biological correctness of the inferred mechanisms remains unclear. Docking and contact comparisons provide indirect evidence of functional relationships, and the rationale evaluations do not directly verify whether the proposed mechanisms are correct for individual molecules.
Questions for authors
  1. How is the ambiguity interval for triggering additional docking analysis defined, and how sensitive are the results to its choice?
  2. Could the authors clarify the inconsistencies in the expert evaluation? Section 4.3.2 reports 70.8% first-place rankings and no last-place rankings, while Figure 5 shows 80.3% and 6.8%, respectively. In addition, Figure 4's caption mentions 40 molecules, whereas the text and Appendix B.3 report 20.
5 comments
AuthorsOpenReview ↗

Response to Reviewer oNJY (1/3)

We thank the reviewer for the constructive feedback and specific suggestions. Below, we summarize the points raised, followed by our responses and discussion of each point.

Point.1: Comparisons with Established Molecular Property Predictors‾\color{red}{\underline{\textsf{Point.1: Comparisons with Established Molecular Property Predictors}}}

Point.2: Contribution of Mechanism-Guided Reasoning‾\color{purple}{\underline{\textsf{Point.2: Contribution of Mechanism-Guided Reasoning}}}

Point.3: Biological Plausibility of Inferred Mechanisms‾\color{green}{\underline{\textsf{Point.3: Biological Plausibility of Inferred Mechanisms}}}

Point.4: Sensitivity Analysis of the Ambiguity Interval‾\color{blue}{\underline{\textsf{Point.4: Sensitivity Analysis of the Ambiguity Interval}}}

Point.5: Consistency of Expert Evaluation Results‾\color{black}{\underline{\textsf{Point.5: Consistency of Expert Evaluation Results}}}

Detailed responses to each comment are provided below.


[Point.1] Comparisons with Established Molecular Property Predictors‾\color{red}{\underline{\textsf{[Point.1] Comparisons with Established Molecular Property Predictors}}}

We agree that our LLM-focused comparisons provided limited coverage of conventional supervised predictors and pretrained molecular models. We therefore added Random Forest, XGBoost, Chemprop, GROVER, MolE, Uni-Mol, and ChemFM. Conventional predictors were trained on the UniTox training set, while pretrained models underwent task-specific fine-tuning on the same set. All models were evaluated on the same evaluation set, with no overlap between training and evaluation molecules.

Table.1 Comparison with supervised and pretrained molecular baselines (Macro-F1).

ModelCirc.Metab.Sens.Repro.Average
Supervised Baselines
RF0.4940.4870.3150.5210.454
XGBoost0.4600.4720.4540.5140.475
Chemprop0.4980.5100.3150.4280.438
Pre-trained Molecular Models
GROVER0.4170.4220.3150.3260.370
MolE0.4510.5110.3150.4230.425
Uni-Mol0.4270.4610.3150.4280.408
ChemFM0.5240.5190.3270.3260.424
Multi-Agents
PharmLens (ours)0.5160.5540.5830.5310.546

As shown in the Table 1, PharmLens achieves the highest Average Macro-F1 and leads in three of the four toxicity groups. Under this evaluation setting, its advantage thus extends beyond LLM-based baselines to both conventional supervised predictors and task-fine-tuned molecular models.

These findings, consistent with our ablations and specification swaps, suggest that mechanism-guided integration of molecule-specific physicochemical and protein–relation evidence adds discriminative information beyond conventional structure–property learning. Additional results and implementation details are included in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer oNJY (2/3)

[Point.2] Contribution of Mechanism-Guided Reasoning‾\color{purple}{\underline{\textsf{[Point.2] Contribution of Mechanism-Guided Reasoning}}}

In PharmLens, QSAR-based relation evaluation is an integral part of mechanism-guided reasoning, not an independent alternative to it. QSAR ensembles assess the molecule–protein relations specified by pharmacological knowledge rather than directly predicting the final endpoint. The same mechanism specification guides physicochemical assessment, selective docking, and the endpoint-specific interpretation of the acquired evidence.

Nevertheless, we agree with the reviewer that the added value of the proposed integration warrants further assessment. We therefore evaluated XGBoost and MLP using ECFP4 augmented with the three evidence sources acquired by PharmLens (ELCE(\textbf{E}_{\text{LCE}}, EPCE\textbf{E}_{\text{PCE}}, EIE)\textbf{E}_{\text{IE}}) without the proposed reasoning workflow. Each model was compared with its ECFP4-only counterpart under the same UniTox evaluation protocol as PharmLens.

Table.2 Comparison with same-evidence baselines on the UniTox (Values in parentheses indicate relative balanced accuracy improvements (%) over the corresponding ECFP4 baseline.)

ModelCirculatoryMetabolicSensoryReproductiveAvg.
XGBoost (ECFP4)0.55410.53520.56340.60190.5637
XGBoost (ELCE(\textbf{E}_{\text{LCE}}, EPCE\textbf{E}_{\text{PCE}}, EIE)\textbf{E}_{\text{IE}})0.5472 (−1.25\color{#B34A44}(−\mathsf{1.25}\\%)0.5554 (+3.77\color{green}(+\mathsf{3.77}\\%)0.6303 (+11.87\color{green}(+\mathsf{11.87}\\%)0.6300 (+4.67\color{green}(+\mathsf{4.67}\\%)0.5907 (+4.79\color{green}(+\mathsf{4.79}\\%)
MLP (ECFP4)0.50150.52540.48730.52110.5088
MLP (ELCE(\textbf{E}_{\text{LCE}}, EPCE\textbf{E}_{\text{PCE}}, EIE)\textbf{E}_{\text{IE}})0.5045 (+0.60\color{green}(+\mathsf{0.60}\\%)0.5641 (+7.37\color{green}(+\mathsf{7.37}\\%)0.4886 (+0.27\color{green}(+\mathsf{0.27}\\%)0.5467 (+4.91\color{green}(+\mathsf{4.91}\\%)0.5260 (+3.38\color{green}(+\mathsf{3.38}\\%)
PharmLens (ours)0.55470.56880.73340.67330.6326

Both evidence-augmented models improve over their ECFP4-only counterparts on average, yet PharmLens outperforms both. These results demonstrate that the acquired evidence is useful across predictive architectures, while showing that conventional supervised integration of the same evidence does not reproduce PharmLens’s performance under the evaluated conditions. Together with our existing ablations and specification-swap experiments, this comparison supports the utility of evidence acquisition and the additional predictive value of mechanism-guided integration. These results have been incorporated into the revised manuscript.


[Point.3] Biological Plausibility of Inferred Mechanisms‾\color{green}{\underline{\textsf{[Point.3] Biological Plausibility of Inferred Mechanisms}}}

We appreciate the reviewer’s concern about the biological validity of the generated rationales. Our goal, however, is not to establish the causal mechanism of an individual molecule, but to define a literature-based pharmacological mechanism for each property and evaluate each query molecule according to this fixed mechanism, using molecule-specific values as evidence for the molecular factors and functional relations with proteins specified by that mechanism. Unfortunately, obtaining comprehensive molecule-specific experimental evidence for the specified factors and functional protein relations remains challenging at this scale.

To assess the validity of this reasoning as far as the computational setting allows, we assess this reasoning in two ways. Expert and LLM-based rationale evaluation judges the biological and chemical soundness and the logical consistency of the generated reasoning (Fig.4,5 in submission), and the downstream BERT experiment measures its predictive usefulness (Tab.4 in submission). These do not replace experimental confirmation, but they evaluate the content and the informativeness of per-molecule reasoning from two different angles (The biological and chemical plausibility of the rationales generated by PharmLens was assessed on a per-molecule basis in both the expert and LLM-as-a-judge evaluations. The questionnaire used for the expert evaluation is provided in Appendix B.3.)

PharmLens also uses literature-based specifications and an evaluator validated on experimentally derived relation labels, and docking and contact comparison serve as supporting structural evidence rather than as a means of confirming a functional relation; where evidence is insufficient, the relation is kept uncertain. Our contribution therefore lies not in the experimental confirmation of individual mechanisms, but in showing that reasoning which acquires and integrates per-molecule evidence under pharmacological knowledge improves property prediction and explanation quality. We state this scope in the Ethics Statement.

AuthorsOpenReview ↗

Response to Reviewer oNJY (3/3)

[Point.4] Sensitivity Analysis of the Ambiguity Interval‾\color{blue}{\underline{\textsf{[Point.4] Sensitivity Analysis of the Ambiguity Interval}}}

We selected the ambiguity interval empirically based on disagreement among the 12 QSAR models at the relation-prediction stage. Specifically, for each molecule–protein relation instance, we measured disagreement among the individual models’ predicted probabilities and examined the distribution of high-disagreement instances across intervals of the ensemble mean probability. These instances were most concentrated in the 0.3–0.5 range, motivating our choice of ([0.3, 0.5)) as the ambiguity interval.

While the interval was selected based on relation-level disagreement, we assessed its sensitivity in terms of final endpoint prediction performance on UniTox. We evaluated six settings by combining lower bounds of 0.3 and 0.4 with upper bounds of 0.5, 0.6, and 0.7.

Table.3 Sensitivity of PharmLens to ambiguity interval selection.

Interval[0.3, 0.5)[0.3, 0.6)[0.3, 0.7)[0.4, 0.5)[0.4, 0.6)[0.4, 0.7)MeanStd
PharmLens0.56270.55350.55320.53760.54560.54770.55000.0085

Performance ranged from 0.5376 to 0.5627, with a mean of 0.5500 and an across-setting sample standard deviation of 0.0085. While minor variation was observed, predictive performance remained broadly consistent across the evaluated interval settings.


Point.5: Consistency of Expert Evaluation Results‾\color{black}{\underline{\textsf{Point.5: Consistency of Expert Evaluation Results}}}

We apologize for the oversight. The values in Section 4.3.2 reflected an earlier version of the expert evaluation. After additional experts participated, we updated Figure 5 but inadvertently left the corresponding text unchanged. The correct first-place and last-place ranking percentages are 80.3% and 6.8%, respectively. Similarly, the reference to 40 molecules in the caption of Figure 4 was retained from an earlier evaluation setup. The final evaluation included 20 molecules, as correctly stated in Section 4.3.2 and Appendix B.3. We have corrected both discrepancies and uploaded the revised manuscript. The text and figure captions now consistently report the final evaluation settings and results.


We hope our responses and additional results have addressed the reviewer’s concerns. We sincerely appreciate the constructive feedback and would be grateful if the clarifications and new findings could be considered in the updated assessment.

Reviewer_mdiUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PharmLens, a pharmacological mechanism-guided multi-agent framework for molecular property prediction. Unlike conventional approaches that primarily predict molecular properties from chemical structures, PharmLens explicitly incorporates biological mechanisms and molecule–protein interactions to improve prediction accuracy and interpretability, particularly for ADME/T-related properties. The proposed framework consists of a knowledge agent, a physicochemical agent, a prediction agent, a protein-relation agent, and an interaction agent. The authors evaluate PharmLens on four molecular property prediction tasks. Experimental results demonstrate that PharmLens outperforms the evaluated LLM-based baselines. Finally, evaluations using downstream prediction tasks, LLM judges, and human experts suggest that PharmLens generates more informative and higher-quality rationales than the compared approaches.

Critical strengths
  • The paper introduces a pharmacological mechanism-guided approach to molecular property prediction, moving beyond conventional structure-based prediction methods.
  • The proposed PharmLens framework systematically integrates literature-grounded mechanistic knowledge, physicochemical descriptors, protein–relation predictions, and molecular docking.
  • The proposed method has strong empirical performance across multiple benchmarks. The authors also conduct several meaningful experiments, including backbone comparisons, evidence-layer ablations, and mechanism-specification swapping, in order to further investigate the performance of the proposed method.
Critical weaknesses
  • The baseline misses some strong conventional methods, including random forest/xgboost + ECFP fingerprints, Chemprop, GROVER, and Uni-Mol.
  • Although the proposed framework is interesting, its individual components are largely based on existing methods and tools, including RDKit for molecular descriptors, conventional machine learning models for protein-relation prediction, UniDock for molecular docking, and LLMs for evidence integration. This feels like an integration of all the best components into an agent system.
Questions for authors
  • Can you try using single LLM with the same evidence to show the necessity of the multi-agent framework?
  • Would a conventional machine learning model, such as XGBoost or an MLP, achieve comparable performance when given the same molecular descriptors and protein-relation predictions?
  • How well does PharmLens generalize to novel molecules with limited existing experimental evidence or literature coverage?
  • How sensitive is the framework to the accuracy and completeness of these automatically generated specifications?
6 comments
AuthorsOpenReview ↗

Response to Reviewer mdiU (1/4)

We thank the reviewer for their careful assessment of our work and constructive feedback. Below, we summarize the key points raised and provide a point-by-point response.

Point.1: Originality of the Framework Beyond Existing Components‾\color{purple}{\underline{\textsf{Point.1: Originality of the Framework Beyond Existing Components}}}

Point.2: Benchmarking Against Strong Molecular Prediction Baselines‾\color{red}{\underline{\textsf{Point.2: Benchmarking Against Strong Molecular Prediction Baselines}}}

Point.3: Added Value over Existing Models Given the Same Evidence‾\color{green}{\underline{\textsf{Point.3: Added Value over Existing Models Given the Same Evidence}}}

Point.4: Robustness to Specification Errors and Omissions‾\color{blue}{\underline{\textsf{Point.4: Robustness to Specification Errors and Omissions}}}

Point.5: Performance on Novel Molecules with Sparse Evidence‾\color{black}{\underline{\textsf{Point.5: Performance on Novel Molecules with Sparse Evidence}}}

Detailed responses to each comment are provided below.


Point.1: Originality of the Framework Beyond Existing Components‾\color{purple}{\underline{\textsf{Point.1: Originality of the Framework Beyond Existing Components}}}

We agree that PharmLens uses established tools, including RDKit, conventional ML models, and Docking tool. Our contribution is to organize property prediction around explicit, molecule-specific assessments of molecular factors and protein–functional relations implicated in the endpoint’s biological mechanisms. These assessments provide an intermediate step between molecular input and endpoint prediction, with the acquired evidence interpreted according to each factor’s biological role (§3.1–3.5).

Applying these tools requires deciding which molecular characteristics and functional relations to assess and what their outputs imply for the target property. PharmLens constructs a property-specific mechanism specification to guide these decisions, acquires the corresponding evidence through tool-based calculations and predictions, and integrates it according to its mechanistic relevance. This automated connection from pharmacological knowledge to assessment-target selection, evidence acquisition, and endpoint prediction is a central methodological contribution of our framework.

PharmLens’s novelty therefore lies in translating pharmacological knowledge into an automated assessment and reasoning procedure, with established tools supporting its execution. We would appreciate the reviewer’s consideration of this methodological contribution when evaluating the originality of our framework.


Point.2: Benchmarking Against Strong Molecular Prediction Baselines‾\color{red}{\underline{\textsf{Point.2: Benchmarking Against Strong Molecular Prediction Baselines}}}

We acknowledge that our original evaluation, which focused on LLM-based approaches, did not sufficiently cover conventional supervised predictors and pretrained molecular models. To broaden the comparison, we added Random Forest, XGBoost, Chemprop, GROVER, MolE, Uni-Mol, and ChemFM. The conventional supervised baselines were trained on the UniTox training set, while the pretrained molecular models were fine-tuned for the target tasks using the same training data. Every model was assessed on an identical evaluation set, with no molecules shared between the training and evaluation sets.

Table 1. Macro-F1 comparison with supervised baselines and pretrained molecular models.

ModelCirc.Metab.Sens.Repro.Average
Supervised Baselines
RF0.4940.4870.3150.5210.454
XGBoost0.4600.4720.4540.5140.475
Chemprop0.4980.5100.3150.4280.438
Pre-trained Molecular Models
GROVER0.4170.4220.3150.3260.370
MolE0.4510.5110.3150.4230.425
Uni-Mol0.4270.4610.3150.4280.408
ChemFM0.5240.5190.3270.3260.424
Multi-Agents
PharmLens (ours)0.5160.5540.5830.5310.546

Table 1 shows that PharmLens obtains the highest average Macro-F1, with the best performance in three of the four toxicity groups. Within this evaluation setting, the expanded comparison demonstrates an overall advantage over conventional supervised predictors and task-specific fine-tuned molecular models, extending the results previously observed against LLM-based baselines.

Together with our evidence ablations and specification swaps, these results suggest that mechanism-guided integration of molecule-specific physicochemical and protein–relation evidence adds discriminative information beyond conventional structure–property learning. The additional results and implementation details are included in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer mdiU (2/4)

Point.3: Added Value over Existing Models Given the Same Evidence‾\color{green}{\underline{\textsf{Point.3: Added Value over Existing Models Given the Same Evidence}}}

To assess whether PharmLens outperforms single-LLM baselines given the same evidence, we evaluated four single LLM baselines—Llama-3.1-8B, GPT-4o, DeepSeek-V4-Pro, and Qwen3.7-Max—with the same evidence acquired by PharmLens. Each evidence-augmented model was compared with its standalone counterpart on the same UniTox set.

Table 2. Single-LLM baselines with and without the same evidence on the UniTox (Macro-F1). Values in parentheses indicate relative Macro-F1 improvements (%) over the corresponding standalone LLM.

ModelCirc.Metab.Sens.Repro.Mean
Llama-3.1-8B0.37320.44720.40380.36370.3968
Llama-3.1-8B + same evidence0.4562 (+22.24\color{green}(+\mathsf{22.24}\\%)0.4904 (+9.66\color{green}(+\mathsf{9.66}\\%)0.5402 (+33.78\color{green}(+\mathsf{33.78}\\%)0.4682 (+28.73\color{green}(+\mathsf{28.73}\\%)0.4888 (+23.19\color{green}(+\mathsf{23.19}\\%)
GPT-4o0.45810.50310.51310.41020.4711
GPT-4o + same evidence0.4701 (+2.62\color{green}(+\mathsf{2.62}\\%)0.5463 (+8.59\color{green}(+\mathsf{8.59}\\%)0.5431 (+5.85\color{green}(+\mathsf{5.85}\\%)0.5031 (+22.65\color{green}(+\mathsf{22.65}\\%)0.5156 (+9.45\color{green}(+\mathsf{9.45}\\%)
DeepSeek-V4-Pro0.42760.46970.47900.48750.4660
DeepSeek-V4-Pro + same evidence0.4569 (+6.85\color{green}(+\mathsf{6.85}\\%)0.5100 (+8.58\color{green}(+\mathsf{8.58}\\%)0.5456 (+13.90\color{green}(+\mathsf{13.90}\\%)0.5059 (+3.77\color{green}(+\mathsf{3.77}\\%)0.5046 (+8.28\color{green}(+\mathsf{8.28}\\%)
Qwen3.7-Max0.42570.47710.51550.51460.4832
Qwen3.7-Max + same evidence0.4492 (+5.52\color{green}(+\mathsf{5.52}\\%)0.4930 (+3.33\color{green}(+\mathsf{3.33}\\%)0.5456 (+5.84\color{green}(+\mathsf{5.84}\\%)0.5205 (+1.15\color{green}(+\mathsf{1.15}\\%)0.5021 (+3.91\color{green}(+\mathsf{3.91}\\%)
PharmLens (ours)0.51560.55400.58270.53080.5458

Evidence augmentation improves every single-LLM baseline across all groups, demonstrating the broader applicability of PharmLens’s evidence across the evaluated LLM backbones. PharmLens nevertheless retains an advantage in every group, including over the evidence-augmented single LLM using the same backbone. These results indicate that evidence availability alone does not account for the full improvement and support the additional value of PharmLens’s mechanism-guided evidence assessment and integration under the evaluated conditions.

AuthorsOpenReview ↗

Response to Reviewer mdiU (3/4)

Point.4: Robustness to Specification Errors and Omissions‾\color{blue}{\underline{\textsf{Point.4: Robustness to Specification Errors and Omissions}}}

Recognizing the need to assess sensitivity to specification quality, we conducted a mechanism-specification swap experiment and reported it in the original submission (§4.2). We held the target endpoint, evaluation molecules, backbone, and tools fixed, replaced the task-matched specification with one from another endpoint, and reacquired evidence accordingly. This intervention measured, at the level of the entire specification, how changes in specification-defined assessment targets and interpretation rules affect the final prediction.

Across all seven evaluated endpoints, mean Macro-F1 over the alternative specifications was 5.5–29.2% lower than under the task-matched condition (Fig. 3). These results quantify the importance of appropriate assessment targets and interpretation rules, supporting our approach of translating pharmacological knowledge into an explicit procedure for acquiring and interpreting molecule-specific evidence.

To further characterize sensitivity to partial errors and omissions, we conducted additional experiments, separately evaluating assessment-item omission and incorrect protein selection within the same endpoint’s specification.

For the removal experiment, we randomly removed 50% of the assessment items from each endpoint’s specification.

Table 3. Sensitivity to specification incompleteness on UniTox (Macro-F1).

All Test MoleculesProperty-cliff Molecules
Circ.Metab.Sens.Repro.AverageCirc.Metab.Sens.Repro.Average
PharmLens (full)0.50660.55810.61930.67000.58850.51560.55400.58270.53080.5458
PharmLens (half)0.48450.51930.56160.56880.53360.42230.40970.55010.50090.4707
Relative decrease (%)4.366.959.3215.109.3318.1026.055.595.6313.76

On UniTox, overall Macro-F1 decreased on both the full test set and the property-cliff subset, with performance declining across all four physiological-system groups in both evaluations. These results indicate that specification completeness contributes to predictive performance, with a larger overall decline on property-cliff molecules. This suggests that broad coverage of mechanistic assessment items may be particularly important for distinguishing structurally similar molecules with different toxicity outcomes. Omitting specification items may limit the acquisition of evidence needed to resolve these differences.

Next, to assess sensitivity to specification correctness, we replaced the relevant proteins with proteins from other endpoints’ specifications while keeping the remaining specification content unchanged, and acquired relation and structural evidence for the substituted proteins.

Table 4. Sensitivity to specification incorrectness on UniTox (Macro-F1).

All Test MoleculesProperty-cliff Molecules
Circ.Metab.Sens.Repro.AverageCirc.Metab.Sens.Repro.Average
PharmLens (original specification)0.50660.55810.61930.67000.58850.51560.55400.58270.53080.5458
PharmLens (perturbed specification)0.47500.52730.58540.57430.54050.43920.45160.52870.47810.4744
Relative decrease (%)6.245.525.4814.278.1614.8218.499.279.9213.08

Macro-F1 decreased across all four groups on both the full test set and the property-cliff subset. These results suggest that incorrect information in the specification can misdirect evidence acquisition and undermine the intended mechanistic assessment, thereby reducing predictive accuracy. Consistent with the omission experiment, the larger decline on the property-cliff subset highlights the potential sensitivity of distinguishing structurally similar molecules with different toxicity outcomes to such errors.

These experiments show that specification completeness and correctness matter for prediction, underscoring the need to address both when constructing specifications. Our literature-grounded construction procedure supports specification quality review: each claim is linked to its source and supporting passage, with the scope of support, applicability conditions, and limitations recorded alongside it. This enables claims with insufficient or potentially conflicting support to be identified and revised, facilitating the review and improvement of specification quality.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.