- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
PharmAgent is a Generator/Reviewer loop built around a frozen LLM for molecular optimization under a fixed oracle budget. Three external-state mechanisms are claimed: a Lagrangian controller with a quadratic penalty that updates multipliers from the accepted state; Chemical-CER, which retrieves past transitions by Morgan fingerprint proximity; and a curriculum that activates constraints at fixed round boundaries. Evaluation covers PMO (E1) and ADMET-PMO (E2) across five tasks and five runs at 1,000 unique oracle evaluations. The headline claims are a 37.3% gain in summed target AUC over MOLLEO and a 53.8% gain in AA-AUC over ExLLM.
Critical strengths
The separation of current measurements from accumulated multiplier pressure in the prompt—together with the observation that a feasible molecule can still carry substantial pressure, thereby raising the cost of reintroducing a violation—constitutes a substantive conceptual contribution. It is a genuine contribution that merits defense under a parity comparison.
Critical weaknesses
-
Six checks are enabled by default and applied conjunctively—Lipinski, Veber, PAINS, Brenk, SA ≤ 6.0, and QED ≥ 0.3—thereby discarding candidates before any oracle call. Four of these thresholds match the “easy” constraints in Table A4 exactly, yet the paper describes this procedure as validity checking, and Appendix F states that PAINS, Brenk, and Veber are never used as optimization guidance. This claim is empirically testable: if the gate is active, the 70.87% infeasible fraction reported in Appendix H should fall entirely within hERG, CYP2D6, and Caco-2. If that is the case, E1 is not target-only, the “0 → 4 → 7” schedule does not describe the implementation, and the E2−E1 contrast no longer constitutes a contrast in online constraint control.
-
With non-decreasing multipliers, boundedness and summability become equivalent assertions; the observed maximum of 1815.53 demonstrates that the premise fails substantively. The abstract selects the most favorable comparator for each metric, and the claim to “rank first among all evaluated methods” is false at the task level. SEISMO, evaluated with only 6–47 evaluations versus 1,000, should not share a ranking column with best/second-best marks. Initialization is unmatched—a curated 26-molecule pool versus ZINC—under a metric in which early feasibility supplies most of the area. On Albuterol, the target equals 1.000 at both endpoints although neither state is albuterol by mass, a saturation effect that merits discussion. The 1,000-call regime structurally disadvantages population methods and should be acknowledged in the limitations. Oracle calls are matched, but LLM computation is not: 420M tokens per E2 arm versus unreported baselines. A single closed-provider backend makes the reported numbers unreproducible in principle.
Questions for authors
-
Does the MedChem-RAG corpus overlap with the training datasets of the GSK3β, JNK3, and DRD2 oracles? If a deduplication protocol exists, please provide it, together with the E1 results obtained using a corpus cleaned of these target genes.
-
Please report confidence intervals for each row of Table 4, as well as the results of reversing the task execution order when CER is active (reversing the task order should suffice).
-
How sensitive are PHV and the resulting ranking method to the scales ri and steepness si reported in Table A5?