- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes an RL-based approach for SBDD. It features a REINVENT-inspired chemical language model (CLM) generator trained freshly for each protein target, and a data-augmentation framework that employs GPT-4o to propose molecular modifications. The LLM is given the CLM-generated molecules' structure and tasked to improve the molecule. The resulting modified molecules are added to the replay buffer as additional training examples for the CLM.
Critical strengths
The framework offers an interesting mechanism for transferring LLM-proposed molecular improvements into molecule generator training. I believe this data augmentation mechanism could be useful if implemented correctly.
Critical weaknesses
- Table 1 lacks modern diffusion- and flow-based models for SBDD (like AliDiff or DrugFlow), RL-based molecule generators FREED and FREED++, and GFN-based generators such TacoGFN, SynFlowNet, RxnFlow. Main exp (table 3) does not contain adequate baselines. It is wrong to compare a CLM that is trained per-target to optimize docking score with Target/DecompDiff and Pocket2Mol which are trained to replicate the training dataset with no reward optimization. Appropriate baselines would be: FREED/FREED++, REINVENT, TacoGFN, SynFlowNet, RxnFlow, CGFlow, AliDiff, DrugFlow. Without this comparison, we cannot establish the scientific value of the proposed method.
- The LLM is basically used to augment training dataset for the CLM model. With the current set of experiments it is impossible to determine whether the stated improvement comes from insights provided by the LLM or from merely better exploration. To establish this, an ablation is needed. To begin with, the authors should swap the LLM augmentation for some kind of random augmentation (i.e., the combinatorial generator from FREED++). Another ablation could be to train a CLM model with better exploration. However, taking into consideration that it contributes a maximum of only 300 molecules (with the total budget of 20000 molecules per run), I find it hard to believe that it can improve the generator in any meaningful way.
Other weaknesses
- I believe the per-protein setting is much less useful than the setting of modern GFN/diffusion/flow-based models as it requires training and (in case of POSEIDON) LLM inference on every new protein target. Moreover, the AutodockVina docking score and SA are not really useful in real drug design tasks and correlate poorly with experimental data. This is especially relevant in Section 4.2 where the proposed method is used to design binders to a specific target. Such binders must certainly be evaluated with other chemical software (such as FEP or at least Boltz or similar model) in order to demonstrate the adequacy of generated molecules.
- There is a number of poorly motivated mechanisms that lack ablations: selection of candidates for editing (eq. 6), the bandit-style prompt selection (eq. 7). Moreover, there are no ablations on the information given to the LLM to perform modifications.
- The paper does not read fluently as it contains repeated statements (i.e., that CLM samples and LLM structural edits are evaluated by the same docking pipeline), over-complicated explanations of simple concepts, and dispensable mathematical notation used only once throughout the paper.
- «Optional prompt evolution» mentioned in contributions; line 265 also mentions «prompt evolution». However, the prompt evolution is explicitly disabled in the main experiment and no separate evaluation of this feature is reported.
Questions for authors
Clarifications
- Given that the individual characteristics exhibit little change after adding the LLM to the training pipeline, what causes such an increase in joint success? Can It be the effect of decreased diversity?
- As I understand, the molecules that are added to the training data by the LLM are also used to select top performing molecules during evaluation. It would be interesting to compare the molecules proposed by the LLM and the molecules proposed by the CLM in terms of docking score.
- «...initialized from a pretrained prior π0» — where does this prior come from?
Suggestions
- It would be interesting to see if a stronger LLM with agentic capabilities can deliver the same type of edits without the extensive feature engineering.