- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes an LLM-based framework for interpreting machine learning (ML) toxicity predictions and feature importance results to support drug design specialists. Using a PubChem cytotoxicity assay dataset, the authors train 10 ML models and generate predictive performance metrics and feature importance scores. These results are then provided to four LLMs, which generate interpretations across six predefined tasks using three prompt variants. The generated responses are evaluated through a three-tier framework consisting of deterministic evaluation, claim and hallucination analysis, and LLM-as-a-judge evaluation. The authors analyze the faithfulness, clarity, and consistency of LLM-generated interpretations across different models, tasks, and prompting strategies.
Critical strengths
Well-motivated research
The paper addresses an important challenge in bridging the gap between ML-based toxicity predictions and their practical use by drug design specialists.
Critical weaknesses
W1. Unclear practical utility for drug design specialists
It is unclear how the proposed framework actually benefits drug design specialists. What specific decisions is the framework intended to support? Is its primary goal to help specialists select among different toxicity prediction models, or to interpret individual toxicity predictions?
W2. Limited evaluation across toxicity endpoints
The experiments are conducted on a single PubChem cytotoxicity assay dataset. Since the framework is intended for toxicity interpretation in drug discovery, the authors should evaluate it across multiple toxicity endpoints and investigate whether the generated explanations provide endpoint-specific insights.
W3. Lack of qualitative analysis
The paper lacks representative case studies demonstrating how the framework generates and interprets toxicity predictions. Providing examples of LLM-generated explanations, including both successful and failure cases, would help readers better understand the framework's behavior and assess the quality and practical usefulness of its outputs.
W4. Lack of human evaluation
Despite targeting drug design specialists, the paper does not include evaluations by domain experts. Although the authors mention human evaluation as future work, I believe it is essential for this study to demonstrate whether the generated explanations are actually useful for the intended users, rather than leaving it for future work.
W5. Missing overview figure
The paper lacks a main figure illustrating the proposed framework. An overview figure showing the complete pipeline, from ML toxicity predictions to LLM-generated explanations and their evaluation, would improve the clarity of the methodology.
Questions for authors
Please refer to the weaknesses listed above for my main questions and concerns.
Additionally, I could not access the GitHub repository linked in the Reproducibility Statement, and the supplementary materials were also unavailable. Therefore, I was unable to inspect the code or verify the implementation details. Could the authors provide accessible links to the code and supplementary materials?