acceptodds
Under review as a conference paper at ICLR 2027

POSEIDON: Reinforcement Learning for Molecular Design with LLM-Guided Structural Feedback

Abstract

Reinforcement learning (RL) is a powerful approach to goal-directed molecular generation, yet docking-driven workflows commonly reduce protein–ligand evaluations to scalar rewards. This compression leaves rich information in docked poses underutilized, including spatial complementarity, interaction patterns, and local structural deficiencies that could guide molecular improvement. We propose POSEIDON, a framework that integrates a frozen large language model (LLM) into the learning loop of an RL-based chemical language model (CLM). POSEIDON translates docking-derived structural evidence into explicit, interpretable molecular edits, re-evaluates the resulting candidates through docking, and incorporates verified outcomes into a shared candidate memory. These candidates are replayed to update the CLM, allowing structure-guided improvements to influence subsequent molecular generation. To make this feedback loop adaptive and cost-efficient, POSEIDON combines three components: an evolving domain-specific prompt informed by verified editing outcomes, a budget-aware allocation strategy that prioritizes promising opportunities for LLM intervention, and a source-aware memory and replay mechanism that balances candidate quality, chemical diversity, and exploration. Together, these components connect structural reasoning, targeted editing, and policy learning within a unified optimization workflow. We evaluate POSEIDON on the CrossDocked2020 benchmark and a bile acid molecular design case study, examining optimization efficiency, molecular diversity, and adherence to structural constraints.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Molecular Design, Reinforcement Learning, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Arn3OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an RL-based approach for SBDD. It features a REINVENT-inspired chemical language model (CLM) generator trained freshly for each protein target, and a data-augmentation framework that employs GPT-4o to propose molecular modifications. The LLM is given the CLM-generated molecules' structure and tasked to improve the molecule. The resulting modified molecules are added to the replay buffer as additional training examples for the CLM.

Critical strengths

The framework offers an interesting mechanism for transferring LLM-proposed molecular improvements into molecule generator training. I believe this data augmentation mechanism could be useful if implemented correctly.

Critical weaknesses
  • Table 1 lacks modern diffusion- and flow-based models for SBDD (like AliDiff or DrugFlow), RL-based molecule generators FREED and FREED++, and GFN-based generators such TacoGFN, SynFlowNet, RxnFlow. Main exp (table 3) does not contain adequate baselines. It is wrong to compare a CLM that is trained per-target to optimize docking score with Target/DecompDiff and Pocket2Mol which are trained to replicate the training dataset with no reward optimization. Appropriate baselines would be: FREED/FREED++, REINVENT, TacoGFN, SynFlowNet, RxnFlow, CGFlow, AliDiff, DrugFlow. Without this comparison, we cannot establish the scientific value of the proposed method.
  • The LLM is basically used to augment training dataset for the CLM model. With the current set of experiments it is impossible to determine whether the stated improvement comes from insights provided by the LLM or from merely better exploration. To establish this, an ablation is needed. To begin with, the authors should swap the LLM augmentation for some kind of random augmentation (i.e., the combinatorial generator from FREED++). Another ablation could be to train a CLM model with better exploration. However, taking into consideration that it contributes a maximum of only 300 molecules (with the total budget of 20000 molecules per run), I find it hard to believe that it can improve the generator in any meaningful way.

Other weaknesses

  • I believe the per-protein setting is much less useful than the setting of modern GFN/diffusion/flow-based models as it requires training and (in case of POSEIDON) LLM inference on every new protein target. Moreover, the AutodockVina docking score and SA are not really useful in real drug design tasks and correlate poorly with experimental data. This is especially relevant in Section 4.2 where the proposed method is used to design binders to a specific target. Such binders must certainly be evaluated with other chemical software (such as FEP or at least Boltz or similar model) in order to demonstrate the adequacy of generated molecules.
  • There is a number of poorly motivated mechanisms that lack ablations: selection of candidates for editing (eq. 6), the bandit-style prompt selection (eq. 7). Moreover, there are no ablations on the information given to the LLM to perform modifications.
  • The paper does not read fluently as it contains repeated statements (i.e., that CLM samples and LLM structural edits are evaluated by the same docking pipeline), over-complicated explanations of simple concepts, and dispensable mathematical notation used only once throughout the paper.
  • «Optional prompt evolution» mentioned in contributions; line 265 also mentions «prompt evolution». However, the prompt evolution is explicitly disabled in the main experiment and no separate evaluation of this feature is reported.
Questions for authors

Clarifications

  • Given that the individual characteristics exhibit little change after adding the LLM to the training pipeline, what causes such an increase in joint success? Can It be the effect of decreased diversity?
  • As I understand, the molecules that are added to the training data by the LLM are also used to select top performing molecules during evaluation. It would be interesting to compare the molecules proposed by the LLM and the molecules proposed by the CLM in terms of docking score.
  • «...initialized from a pretrained prior π0» — where does this prior come from?

Suggestions

  • It would be interesting to see if a stronger LLM with agentic capabilities can deliver the same type of edits without the extensive feature engineering.
Reviewer_QY5iOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents POSEIDON, a collaborative framework for structure-based drug design. POSEIDON uses a chemical language model (CLM) to generate molecules and learns from docking rewards, and an LLM to propose molecular edits based on multi-pose structural information. The molecules outputed by LLM are leverage in CLM training through shared memory. Experimental results on CrossDocked2020 demonstrate that POSEIDON improves joint success over its CLM-only and LLM-only variants, and a case study shows locally modified derivatives with improved docking scores.

Critical strengths
  1. The collaborative learning mechanism is well-motivated. Intuitively, incorporating the molecules edited by the LLM into the training process of the CLM can provide an informative training signal for the CLM.

  2. The design of the structural evidence is reasonable.

  3. Experimental results on a representative benchmark show the effectiveness of the proposed framework.

Critical weaknesses
  1. The advantage of the collaborative learning mechanism is not clearly verified. Although the full POSEIDON framework outperforms the CLM-only and LLM-only variants, the comparison does not separate the benefit of additional LLM-generated candidates from that of using these candidates to train the CLM. What is needed is a comparison that keeps both candidate sources but does not use feedback from LLM edits for CLM training.

  2. The contribution of the structural information is not directly evaluated. The reported experiments do not show whether providing docking geometry improves LLM editing compared with using only molecular structures and scores, or whether multiple poses are more useful than a single pose.

  3. Comparisons with closely related methods such as CIDD and LIDDIA are not provided.

Questions for authors

The title in the PDF (“POSEIDON: Collaborative Structure-Based Drug Design with CLM Agents and LLMs”) differs from the that on OpenReview (“POSEIDON: Reinforcement Learning for Molecular Design with LLM-Guided Structural Feedback”). Please clarify and make them consistent.

Reviewer_WgyyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

POSEIDON combines a chemical language model for molecular exploration with an LLM for structure-guided editing, using evaluated edits to update the generator through shared memory and replay. The paper reports improved candidate-discovery performance over its isolated components and presents an FXR derivative-design case study.

Critical strengths
  1. The paper addresses a meaningful problem: connecting efficient molecular exploration with structural information that can guide refinement.

  2. The framework is clearly organized, and the reported gains suggest that collaboration between a molecular generator and an LLM editor can be useful. Independent evaluation of proposed edits also provides a sensible basis for learning from their outcomes.

Critical weaknesses
  1. Insufficiently established methodological novelty. Prior work already covers structure-guided LLM editing and feedback from refined molecules to generative-model training. POSEIDON's potential contribution therefore rests on how these mechanisms are connected, particularly its multi-pose interface and source-aware replay. The paper does not yet demonstrate that these designs address a limitation that a simpler combination of existing methods would fail to resolve.

  2. Insufficient validation of the central collaboration claim. Comparisons with isolated components show an overall benefit but do not establish which proposed mechanisms are necessary. The missing controls are central to assessing the contribution. Moreover, improvements in the selected candidate set do not establish that the learned generator becomes better at producing high-quality molecules on its own. The FXR case does not provide independent evidence for this learning benefit. These gaps motivate my weak-reject recommendation despite the encouraging aggregate results.

Questions for authors
  1. What is POSEIDON's substantive methodological advance beyond combining existing structure-guided LLM editing with generator training and replay?

  2. What evidence shows that the proposed interface and replay design drive the gains beyond the benefit of combining generation with refinement?

  3. Does the collaboration improve the CLM's ability to generate high-quality molecules independently, beyond improving the selected candidate set?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.