- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies how to use historical ADME measurements to train an LLM to select molecules from a given candidate set, with the aim of improving target properties while limiting deterioration in other properties. The authors formulate the task as optimization over a probability distribution on the candidates. They introduce an exact target-distribution construction with per-endpoint non-regression constraints, alongside CPDPO-R, which directly optimizes the utility of the policy after applying a TV controller. At deployment, the model receives the source and candidate molecular structures and the desired directions of property changes. The controller limits deviations from a reference policy, providing a theoretical upper bound on expected regression in each endpoint.
The experiments use the Biogen dataset of 3,521 compounds, primarily constructing four-candidate panels for combinations of HLM, RLM, solubility, and MDR1 endpoints, with scaffold-based data splits. On the RLM-solubility task, CPDPO-R with Qwen3-8B achieves a higher mean gain after applying the controller than the scalar-optimization baseline across five training seeds. The authors also conduct external tests on TDC and OpenADMET. The appendices examine missing measurements, candidate-set size, alternative models, comparisons with molecular models, and structured outputs.
Critical strengths
(1) The paper addresses an important problem in drug discovery. Multiparameter optimization (MPO) requires balancing improvements in some properties against deterioration in others. Choosing among candidate molecules with different strengths and weaknesses across ADME properties is a practical need in drug development. By considering changes in individual properties separately, the paper addresses an issue that a single aggregate score may not fully capture.
(2) The paper provides an explicit framework for candidate selection under multiple ADME constraints. The authors use measured properties of the given candidate molecules to train a selection policy. They impose separate constraints on individual properties and use a controller to limit expected deterioration relative to a baseline policy. This provides a concrete way to use an LLM to assist candidate selection, with explicit rules for handling improvements and deterioration across different properties.
Critical weaknesses
The method’s applicability is constrained by the coverage of its multi-property training data. The main training setup uses measured outcomes for multiple target properties in each candidate panel. In practical ADMET datasets, the compounds measured for different properties often overlap only partially. Suitable training data may therefore be scarce for a given property combination. Changing the optimization targets may also require new training panels. The random-masking experiment in Table 13 is a useful addition, but it examines missing measurements within existing endpoint tasks. The evidence for generalization to properties or property combinations with limited joint measurements remains insufficient.
This concern is particularly relevant when the target properties involve mechanisms that differ substantially from those covered by the training data. If applying the method to these properties still requires collecting new joint measurements across multiple endpoints, the same data shortage would recur as the optimization targets change. I therefore remain concerned about how well the method can generalize under these data constraints and how broadly it can be applied.
Questions for authors
(1) Please discuss how well the method can generalize when experimental data are limited. What do the external tests suggest about its potential and limitations for properties or property combinations that are poorly represented in the training data?
(2) Please discuss why an LLM was chosen for this task and what practical value it adds. For selection from a fixed candidate set, which advantages of the LLM are more supported than non-LLM based models by the current results? Please discuss the trade-offs among computational cost, gains after applying the controller, and the reliability of auxiliary outputs.
(3)I am curious about what the model's reported confidence means and how it is calibrated. The supplementary code uses the balanced-score margin between the best and second-best candidates to construct confidence labels. The Brier score, however, is evaluated against whether the model selects the predefined gold candidate. Why was this training label chosen?
(4) Could the authors present a few concrete molecular examples in the main text or appendix to help readers interpret the results?
4 comments
Q1, continued — generalisation under limited joint measurement
Your concern is structural and we accept it. Our new feasibility measurement sharpens rather than softens it: from two to four endpoints, the fraction of panels admitting no confidently non-regressing action rises from 22.7% to 65.2%, and the fraction where an uncertainty-gated rule abstains entirely from 29.5% to 67.9%. Requiring joint measurement across more endpoints shrinks both the eligible compound set and the feasible action set within each panel. We state this as a scope limitation rather than claim broad generality, alongside the random-masking experiment (Table 13) and the six Biogen/TDC/OpenADMET comparisons.
Q2 — why an LLM, and what it adds
Our new baselines let us answer this quantitatively instead of by assertion, and the answer has two parts.
- For the submitted method, the LLM added nothing measurable over classical
deployable pipelines:
P = .844against QSAR-DS,P = .803against FP-Ranker, with the classical methods cheaper, more seed-stable (.033/.141 vs .268 choice disagreement) and better on the worst endpoint. We say so in General Response §1. - For CPDPO-DP the margin is large and consistent:
+.00045910over the tuned QSAR-DS, CI[+.00037500, +.00054400],P = 1.00000, 1.88x on the controlled metric, and 1.56x–1.94x across four model families. Worst-endpoint regression improves 4.4x (−.00004120 vs the tuned baseline's −.00018240), which matters more here than the mean, since the whole point is not harming any single endpoint. - The cost side, which belongs next to it. CPDPO-DP needs one Qwen3-8B + LoRA forward pass over a ~700-token prompt plus ~60 generated tokens: 0.31 s per panel single-stream on one RTX Pro 5000 (72 GB), or 0.019 s per panel at batch 32. QSAR-DS needs four Morgan fingerprints and two 300-tree Random Forest predictions: 1.8 ms per panel on one CPU core. That is roughly 172x in single-stream wall-clock, about 11x at batch 32 against a single core, and some three orders of magnitude in energy. A 1.88x gain for a 172x cost is a judgement that belongs to the reader, not to us, and we present it that way rather than omitting the denominator as the submission did. For a programme where a single assay plate costs more than a month of this inference, we think the trade is favourable; for high-throughput virtual screening it plainly is not, and we say so.
One boundary we want on the record either way. A measurement-informed desirability
rule — which sees the measured endpoint values rather than predicting them from
structure — reaches .00118642. That is the ceiling for any selection policy on these
panels, and we will state it in the paper rather than leave it implicit. Against it,
the submitted CPDPO-R captured 38% and the best classical deployable pipeline 43%;
CPDPO-DP captures 83%. The remaining 17% is structure-to-endpoint prediction error,
which no selection policy can recover and which bounds this entire line of work,
ours included.
On the reliability of auxiliary outputs: see Q3 — we withdraw the calibration claim.
Q4 — concrete molecular examples
Agreed, and useful enough that we should have done it in the submission. We add a figure with three test panels — chosen to span a clear win, a near-tie, and a case where our method loses — showing source and candidate structures, measured endpoint deltas, the anchor distribution, and the displacement each method spends. All three are from the shipped test split (seed 37 policies):
| panel | source SMILES | anchor pick / gold | CPDPO-R TV | Scalar TV | CPDPO-R gain | Scalar gain | CPDPO-R eff. | Scalar eff. |
|---|---|---|---|---|---|---|---|---|
multi2-00001681 clear win | Cc1ccnc(NC(=O)CN2CCCC2=O)c1 | B / C | .1756 | .0584 | +.05157 | −.00516 | +.2937 | −.0885 |
multi2-00000568 near-tie | O=C(c1cc(-c2cccc(Cl)c2)on1)N1CCC(n2cccc2)CC1 | A / A | .1445 | .0958 | +.02975 | +.01969 | +.2059 | +.2056 |
multi2-00001529 we lose | Fc1cccc(CCNCc2cc(-c3ccccc3)cn3nnnc23)c1 | D / C | .4957 | .2056 | −.00705 | +.00054 | −.0142 | +.0026 |
The third panel is the one we think is most informative, so we will present it rather
than only the favourable cases. There CPDPO-R spends 2.4x Scalar's displacement
(.4957 vs .2056) and ends with a negative raw gain; the controller then scales
that loss by eps/t_x and the panel contributes negatively. It is the per-panel
instance of the aggregate mechanism in General Response §2, and it is exactly the
behaviour CPDPO-DP is designed to remove.
The near-tie panel makes the reporting-unit point concretely: the two methods have
almost identical gain per unit displacement (.2059 vs .2056) while their raw gains
differ by 51% (+.02975 vs +.01969), because CPDPO-R also moves 51% further. Raw
reward and deployed gain genuinely are different orderings, and this single panel shows
why.
I agree with the other reviewer that the rebuttal should not simply reproduce AI-generated text; the authors should at least revise passages that read as unedited AI output.
Response to Reviewer q7Qq
Thank you for recording soundness 3, and for Q3, which identifies a real defect we had not noticed.
Q3 — the confidence label and the Brier target
You are right, and the mismatch is three-way rather than two-way. The code contains three distinct objects:
prepare_multi_admet.py:161,rebuild_normalized_panels.py:129):confidence = clip(0.5 + 0.45 * (sorted_balanced[-1] - sorted_balanced[-2]), 0.5, 0.95)— an affine map of the balanced-score margin between best and second-best candidate, clipped to[0.5, 0.95]. A property of the panel, with no reference to any model. This is what you identified.evaluate_action_scores.py:139):confidence = p[choice_index]— the policy's probability mass on its own chosen action. A property of the policy.evaluate_action_scores.py:140):brier = (p[choice_index] - 1{choice == gold})^2— a property of the model's correctness.So there are two defects. The quantity we train is never the quantity we score: the Brier score uses the action probability, not the emitted
confidencefield. And even if it were scored it could not be calibrated: a target clipped to[0.5, 0.95]cannot representP(correct)outside that range, so a Brier score against a 0/1 correctness label is mis-calibrated by construction.We drop any claim that the emitted confidence is calibrated, relabel the margin field as what it is — a panel-difficulty annotation in the supervised continuation — and report reliability for the action-probability predictor only, recalibrated against
P(choice == gold)with an isotonic fit on validation and evaluated on test:For reference, top-1 agreement with the gold candidate on the same split is .4712 for CPDPO-DP against .3797 for QSAR-DS and .3831 for FP-Ranker — all well above the .25 chance rate, and all far from a solved task, which is the honest context for any calibration claim. The recalibration is a fix to the reporting; it does not change any policy or any controlled-gain number in this response.
Q1 and Q2 together — transfer to an endpoint pair never trained on
Your Q1 and Q2 turn out to have a common answer that the submission does not contain, and it changes what we think the LLM is actually for.
We trained on (RLM clearance, solubility) and deployed zero-shot on (HLM clearance, MDR1 efflux) — a different endpoint pair, never seen during training, with only the endpoint directions changed in the prompt. Test panels from the four-endpoint Biogen construction, same controller, same
eps = .005:The last row is the substantive point, and it is structural rather than empirical. A QSAR-plus-desirability pipeline needs one fitted regressor per endpoint; presented with a property combination it was not trained on, it has nothing to predict with, and recovering it requires collecting joint measurements on the new pair — exactly the scarcity you identify in Q1. The LLM policy reads the endpoint directions from its prompt, so it can be asked about a combination that has never been measured jointly, and here it retains 63% of its in-domain performance (
.00061400against.00098200) while still exceeding the retrained classical pipeline's.00049200.So our answer to Q2 changes. The LLM's advantage is not mainly the 1.88x on the in-domain metric, which is real but modest against a cheap and stable baseline. It is that the deployable alternative does not transfer across property combinations at all, and transfer is precisely the regime your Q1 is about. We will reframe the paper's motivation accordingly: the case for a language-model policy here rests on zero-shot specification of new endpoint pairs, not on beating a Random Forest on a pair that is already well measured.
And our answer to Q1 is no longer only a scope concession. The data shortage you describe is real and we quantified how fast it bites (22.7% → 65.2% of panels infeasible from two to four endpoints). But transfer is the mechanism by which a method can be useful despite it, and it is testable, and we had not tested it.