acceptodds
Under review as a conference paper at ICLR 2027

MOLFRA: Online Alignment of Multimodal Molecular Flows via Forward-Process Reinforcement Learning

Abstract

Pretrained generative models for structure-based drug design (SBDD) learn structural regularities from protein–ligand complexes without directly optimizing task-specific molecular properties. Controllable approaches pursue these objectives through sampling-time guidance or iterative optimization, but improving properties while retaining molecular diversity remains a challenge. We introduce **MOLFRA**, an online forward-process reward-alignment method that jointly fine-tunes pretrained multimodal molecular flows. Under a shared pocket-relative reward, MOLFRA couples implicit positive and negative prediction branches in coordinate endpoint and atom-type logit spaces, entirely avoiding sampling-trajectory likelihood ratios or backpropagation through the sampler. On CrossDocked2020, post-training MOLFORM with a joint Vina Score and synthetic accessibility (SA) reward significantly improves mean Vina Dock (from −7.50 to −9.25 kcal/mol) and normalized SA (from 0.60 to 0.74), while fully preserving the pretrained baseline’s molecular diversity (0.78 vs. 0.78).

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Structure-Based Drug Design, Flow Matching, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QMYcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces MOLFRA, an online post-training method for pocket-conditioned multimodal molecular flows. It uses forward-process reconstruction with paired positive and negative branches to jointly align coordinate and atom-type generation under shared pocket-relative rewards. Experiments are mainly conducted on CrossDocked2020 with MOLFORM as the backbone, mainly using SA and vina-based docking score for evaluations.

Critical strengths

The proposed method is generally clear and easy to follow. The overall framework, training procedure, and the roles of different components are explained in a relatively straightforward manner, making the main idea easy to understand.

Critical weaknesses
  1. The methodological novelty may be limited. This work appears to be largely established upon the previous work, such as its core learning rationales and continuous coordinate construction, and then performs positive/negative branching for atom types. It is unclear if there are any substantial new rationales, frameworks, theories, or insights introduced.

  2. Several major concerns for experiments. (1) The key performance metrics overlap with the training reward (e.g. vina score & SA), which may have risks of reward hacking. A good vina score while extremely high strain energy, and abnormal ring-size distributions may further imply this concern, potentially meaning obviously abnormal structures in generated molecules. (2) The evaluation protocol may have severe bias. Each pocket only generates 100 molecules, which may be insufficient. Most of the key metrics are computed based on valid or successfully evaluated molecules, while the overall validity, novelty, uniqueness of generated molecules are not reported. It is possible that there are lots of failed samples but only a few valid molecules with good vina score. Given this, the claim that “fully preserving diversity” may be not supported as well. (3) Baseline comparison may be unfair. In related work, several trajectory-based online RL methods are discussed. However, there is no standard online RL method under the same setting included for comparison. In addition, the results reported in table 1 seem to be cross-paper. The experimental settings may be quite different and not comparable. (4) Experimental scope and reliability are quite limited. The experiments are mainly conducted on just one dataset and backbone. There are also no multi-seed tests or other statistical tests. The optimization tasks may have a relatively high degree of randomness so it is unclear whether the improvements are reproducible.

  3. Potential inconsistency between methods and actual training. (1) Reward-surrogate alignment appears to be established under a fixed rollout-corruption law, posterior calibration, and trainable-anchor agreement. However, the model may keep updating and the EMA anchor may also keep changing during actual training. The assumptions or premises may not always hold during the training. (2) It is unclear if this is an RL method. Although the title includes “RL”, there are no on-policy rollouts from the current model as well as trajectory likelihood ratios, policy gradients, and backpropagation via sampling trajectory, which are commonly used components in the RL framing.

Questions for authors
  1. What are the actual novelties and contributions?

  2. Are there more convincing experimental results that support the claims?

  3. If the premises do not always hold, can the method still work as intended? And could the method be called “RL”?

8 comments
AuthorsOpenReview ↗

Response to Reviewer QMYc (1/4): novelty (W1, Q1), theoretical premises (W3.1, Q3), and the RL framing (W3.2, Q3)

We thank the Reviewer QMYc for the careful review. W1, W3, Q1 and Q3 are answered here; W2 and Q2 are answered in Comments 2 to 4.

W1 & Q1: What are the actual novelties and contributions?

The continuous coordinate branching and the self-normalized loss follow DiffusionNFT, and we do not claim them as new. Our contribution is extending forward-process reward alignment to SBDD generators that couple continuous coordinates with discrete atom types.

  1. Simplex-valid categorical alignment. Negative branching directly on probabilities can leave the simplex. We branch in logit space instead, which gives valid categorical distributions q±∝(q0)1∓βd(qθ)±βdq^{\pm} \propto (q^0)^{1\mp\beta_d}(q^{\theta})^{\pm\beta_d} (Prop. 1). We derive an exact decomposition of the categorical loss into reward-signed feedback and nonnegative anchor-relative KL penalties (Prop. 2).
  2. Joint multimodal optimization. One pocket-relative reward updates coordinates and atom types together, and the pretrained sampler is untouched. In the ablations, coordinate-only and atom-type-only objectives leave Vina Dock close to the pretrained -7.50 (-7.56 and -7.48), while joint alignment reaches -9.25. Removing the negative branch drops Vina Dock to -4.77, below the pretrained model (Table 6).
  3. Validation across SBDD generators. The recipe also improves a diffusion backbone (TargetDiff) and a Bayesian flow backbone (MolCRAFT), and the aligned checkpoints transfer to Binding MOAD without retraining (Tables 3-4).

We therefore do not present a new forward-process RL principle. The contribution is its simplex-preserving multimodal extension, the analysis of categorical feedback, and evidence across different SBDD generators.

W3(1) & Q3: Do the premises hold during training, and can the method still work as intended?

The premises are sufficient for our local theoretical analysis, not prerequisites for applying MOLFRA. We distinguish two levels of results:

  1. Unconditional properties. Propositions 1 and 2 establish simplex validity, the exact decomposition into reward-signed feedback and nonnegative anchor-relative KL penalties, and convexity in the output logits. These hold for any finite logits, without posterior calibration or trainable-anchor agreement.
  2. Local analysis vs. practical training. Proposition 3 establishes categorical reward-surrogate alignment under posterior calibration, trainable-anchor agreement and a fixed rollout-corruption law. The rollout law and the EMA anchor are fixed within each update, but calibration and agreement are not guaranteed. Updates also use finite batches, and the coordinate term can offset the categorical one. Proposition 3 is therefore a local first-order interpretation, not a global improvement guarantee (Appendix E.3).

In practice, we use a small learning rate (5×10−75 \times 10^{-7}), gradient clipping (8.0) and an EMA anchor (τ=0.995\tau = 0.995) to promote stability. Empirical effectiveness is supported independently of the theory by the improvements on three backbones on CrossDocked2020 and the transfer to Binding MOAD (Comment 2).

W3(2) & Q3: Can the method be called "RL"?

Yes, in the sense of DiffusionNFT: online, reward-driven policy optimization. Each round, fresh rollouts from the EMA policy πθold\pi_{\theta_{\mathrm{old}}} are scored with black-box rewards (Vina and SA), and the generator is updated through a reward-dependent forward-process reconstruction objective. Unlike conventional policy-gradient RL, MOLFRA requires neither trajectory likelihood ratios nor backpropagation through sampling (Sec. 3.3). It is also not strictly on-policy with respect to πθ\pi_\theta: samples come from the lagged EMA policy (Sec. 3.2), which acts as the behavior policy in each round of rollout and update. We will clarify this distinction at the first mention of "RL."

AuthorsOpenReview ↗

Response to Reviewer QMYc (2/4): additional backbones and a second benchmark (W2.4, Q2)

We added two pretrained backbones and a second test set. MOLFRA only needs clean-coordinate estimates and atom-type logits, so it transfers across generative paradigms without changing the reward (Vina Score + SA), the pocket-relative normalization (Eq. 2) or the EMA training loop. We apply it to TargetDiff (DDPM with categorical diffusion) and MolCRAFT (Bayesian flow network), each with its own forward corruption and sampler (Section 4.3, Appendix H).

CrossDocked2020 (Table 3; the better value in each pair is in bold):

BackboneModelVina Score ↓Vina Min ↓Vina Dock ↓QED ↑SA ↑Diversity ↑Connected ↑Strain (med.) ↓Clashes ↓
MolCRAFTPretrained-6.15-6.99-7.790.480.660.7296.7%1966.91
+ MOLFRA-6.51-7.65-8.390.510.810.7398.9%826.71
TargetDiffPretrained-5.71-6.43-7.410.490.600.7290.4%120810.67
+ MOLFRA-5.84-6.70-7.590.530.620.7194.1%8059.23

Zero-shot transfer to Binding MOAD (Table 4; 130 pockets from the DiffSBDD split, 100 molecules each; Vina columns are medians over successfully evaluated molecules). We evaluate the CrossDocked2020-trained checkpoints without retraining. The better value in each pair is in bold, and ties are not bolded.

BackboneModelVina Score ↓Vina Min ↓Vina Dock ↓QED ↑SA ↑Connected ↑Strain (med.) ↓Clashes ↓
MolCRAFTPretrained-6.50-6.61-6.970.540.7498.5%1236.21
+ MOLFRA-7.42-7.52-7.870.540.8497.7%814.17
TargetDiffPretrained-5.44-6.04-6.980.530.6383.4%75310.86
+ MOLFRA-5.80-6.32-7.220.560.6585.0%6469.60

The three Vina metrics, SA, median strain and clash count improve on both backbones and both test sets, so the gains are not specific to MOLFORM or to CrossDocked2020.

TargetDiff's sampler needs 1,000 steps, against 100 for MOLFORM and MolCRAFT, so its rollouts take much longer. We therefore gave both new backbones the same wall-clock budget of 20 hours on 4 GB200 GPUs instead of matching the number of rounds: 378 for TargetDiff and 1,428 for MolCRAFT. The pretrained CrossDocked2020 rows repeat the TargetDiff and MolCRAFT rows of Tables 1-2.

On multiple seeds: we did not run multi-seed replicates. The same recipe improves three backbones on CrossDocked2020 and two on Binding MOAD, which makes a seed-specific artifact less likely but does not replace replicates.

AuthorsOpenReview ↗

Response to Reviewer QMYc (3/4): reward hacking, independent scorers, strain filtering and ring sizes (W2.1, Q2)

1. Independent scorers (GNINA Vinardo and CNNaffinity). To test whether the gains carry over to scorers outside the reward, we rescored the generated poses with GNINA, using Vinardo (an empirical scoring function) and CNNaffinity (a 3D CNN affinity predictor trained on experimental binding data):

BackboneModelVinardo ↓CNNaffinity ↑
MOLFORMPretrained-4.585.39
+ MOLFRA-5.055.05
MolCRAFTPretrained-5.125.63
+ MOLFRA-5.315.23
TargetDiffPretrained-4.515.44
+ MOLFRA-5.015.54

The result is mixed. Vinardo improves on all three backbones. CNNaffinity improves on TargetDiff (+0.10) and drops on MOLFORM (-0.34) and MolCRAFT (-0.40). We therefore do not claim the gains are independent of the scorer, and the Limitations section now notes that the gains may be specific to the reward.

2. Are the docking gains driven by high-strain conformations? We tested this two ways.

Strain-filtered docking. We kept only molecules whose PoseCheck strain energy is at most 114 kcal/mol (the median of the experimental reference complexes) and compared mean Vina Dock (kcal/mol):

BackboneCheckpointUnfiltered Vina Dock ↓Strain ≤114\le 114 Vina Dock ↓
MolCRAFTPretrained-7.79-6.64
+ MOLFRA-8.39-8.15
Δ\Delta (MOLFRA −- Pretrained)-0.60-1.51
MOLFORMPretrained-7.50-5.28
+ MOLFRA-9.25-7.23
Δ\Delta (MOLFRA −- Pretrained)-1.75-1.95

The docking advantage over the pretrained base survives the filter, and the margin grows (-0.60 to -1.51 on MolCRAFT, -1.75 to -1.95 on MOLFORM).

Strain and clashes. MOLFRA lowers median strain from 196 to 82.4 on MolCRAFT (native reference: 114) and from 1208 to 804.9 on TargetDiff (Table 3). On MOLFORM, adding SA to the Vina reward cuts median strain by 88%, from 2,139 to 264 (Table 5), and the clash count of 2.28 is the lowest in Table 2.

3. Ring sizes (Appendix G). In Table 7, MOLFRA produces no 3-membered rings (0.0%) and few 4-membered rings (0.6%), and its share of 6-membered rings (68.5%) is close to the reference (66.0%). It does shift mass from 5-membered rings (18.5% vs. 29.6% in the reference) toward 7-membered rings (9.2% vs. 0.8%). Other 3D generators show the same shift (TargetDiff 11.7%, DecompDiff 11.9%, TAGMol 15.0% for 7-rings), and Appendix G already reports it.

Reviewer_kgoQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a post-training method for pretrained generative models for structure-based drug design (SBDD), with the goal of improving targeted molecular properties while retaining the pretrained generator’s diversity. The proposed method, called MOLFRA, iteratively generates and evaluates candidate ligands, then uses their rewards to jointly update coordinate and atom-type generation. An experimental evaluation was performed on the CrossDocked2020 benchmark using MOL FORM as the pretrained generative model. Four other post training methods were compared. The main results, as stated in the Conclusion, is that posttraining MOLFORM improves mean Vina Dock from −7.50 to −9.25 kcal/mol and normalized SA from 0.60 to 0.74, while matching the pretrained base’s reported diversity of 0.78.

Critical strengths
  • Significance of the problem
 Structure-based drug design is an ML application of great practical significance, and the existing method still have a lot of room for improvement. In particular, the synthesizability of generated molecules is often not good enough, see for example
Papidocha, Sven M., et al. "The elephant in the lab: synthesizability in generative small-molecule design." Current Opinion in Chemical Engineering 51 (2026).

  • Solid experimental evaluation using one pretrained generative model
 MOLFRA was tested for the pretrained model MOLFORM and compared against four SOTA post training methods on the CrossDocked2020 benchmark. The authors performed further analyses of the 3D conformation plausibility and structural integrity, important factors for practical applications such as drug discovery.

Critical weaknesses
  • Limitations of the experimental design
 MOLFRA was tested only for one pretrained model, and it remains unclear whether MOLFRA can also improve other pretrained generative models.


  • Unconvincing experimental results
 While MOLFORM significantly improves the Vina Dock score and the SA of MOLFORM, its performance is only on par with that of baseline MolJO.
As pointed out by the authors, MOLFRA attains the clearly lowest PoseCheck clash, but MolJO outerperforms MOLFORM on the other three metrics of 3D conformation plausibility and structural integrity. So overall I see no clear improvement of the state-of-the-art.

Questions for authors
  1. Is MOLFRA also applicable to other pretrained generative models?

  2. If so, how does MOLFRA perform for other pretrained generative models?

4 comments
AuthorsOpenReview ↗

Response to Reviewer kgoQ

We thank Reviewer kgoQ for recognizing the practical significance of the problem and our 3D structural evaluation.

1. Generalizability to other pretrained models (Weakness 1, Q1 & Q2)

Yes. MolFRA only needs clean-coordinate estimates and atom-type logits, so it transfers across generative paradigms without changing the reward, the pocket-relative normalization or the EMA training loop. In Section 4.3 (Tables 3 and 4) and Appendix H, we apply it to TargetDiff (DDPM + categorical diffusion) and MolCRAFT (Bayesian flow network). For each backbone we use its own forward corruption and its own sampler.

CrossDocked2020 (Table 3; better value in each pair in bold; Vina, strain and clashes: lower is better; QED, SA, diversity and Connected: higher is better):

BackboneModelVina ScoreVina MinVina DockQEDSADiversityConnectedStrain (med.)Clashes
MolCRAFTPretrained-6.15-6.99-7.790.480.660.7296.7%1966.91
+ MolFRA-6.51-7.65-8.390.510.810.7398.9%826.71
TargetDiffPretrained-5.71-6.43-7.410.490.600.7290.4%120810.67
+ MolFRA-5.84-6.70-7.590.530.620.7194.1%8059.23

Zero-shot transfer to Binding MOAD (Table 4; 130 pockets, 100 molecules each). We evaluate the CrossDocked2020-trained checkpoints directly, without retraining. Better value in each pair in bold (a tie is not bolded).

BackboneModelVina ScoreVina MinVina DockQEDSAConnectedStrain (med.)Clashes
MolCRAFTPretrained-6.50-6.61-6.970.540.7498.5%1236.21
+ MolFRA-7.42-7.52-7.870.540.8497.7%814.17
TargetDiffPretrained-5.44-6.04-6.980.530.6383.4%75310.86
+ MolFRA-5.80-6.32-7.220.560.6585.0%6469.60

The three Vina metrics, SA, strain and clash count improve on both backbones and both test sets, so the gains are not specific to MolFORM or to CrossDocked2020.

TargetDiff's sampler needs 1,000 steps, so its rollouts take much longer. We therefore trained both new backbones for 20 hours on 4 GB200 GPUs instead of matching the number of steps: 378 rounds for TargetDiff and 1,428 for MolCRAFT. The pretrained CrossDocked rows repeat the TargetDiff and MolCRAFT rows of Tables 1-2.

2. Comparison with MolJO (Weakness 2)

We agree that MolJO is a strong baseline, and we do not claim to dominate it. We now state this explicitly in Section 4.2.

  • Inference paradigm and test-time cost. MolJO performs gradient-guided optimization during inference, requiring repeated evaluations and gradient computations through pretrained differentiable property predictors at each sampling step. MolFRA is a one-time post-training method; the aligned model keeps the pretrained model's fast 100-step sampler and adds no test-time optimization and no gradient computation through property predictors.
  • Same backbone (MolCRAFT, Table 3). MolJO is built on the BFN-based MolCRAFT, so we compare it with MolCRAFT + MolFRA on the same generator. MolJO is better on the Vina metrics, and MolFRA is better on SA, connectivity, strain, clashes and diversity (better value in bold):
ModelVina ScoreVina MinVina DockSAConnectedStrain (med.)ClashesDiversity
MolJO-7.52-8.33-9.050.7897.3%1636.720.66
MolCRAFT + MolFRA-6.51-7.65-8.390.8198.9%826.710.73

In short, MolJO reaches better Vina scores, while MolFRA gives lower strain, higher diversity and no test-time cost. Since MolFRA is a post-training recipe for a pretrained generator, it can also be applied to a stronger base model.

We hope the new results address your concerns.

AuthorsOpenReview ↗

A quick note before the discussion closes

Dear Reviewer kgoQ,

With the discussion period nearly over, we wanted to check whether our response answers your questions. We applied MOLFRA to TargetDiff and MolCRAFT, and we now compare against MolJO on the shared MolCRAFT backbone. If any point needs more detail, we would be glad to address it before the deadline.

Thanks for your thorough rebuttal!

I am glad to see that MOLFRA is applicable to other backbones and that it obtains positive results also for a TargetDiff backbone both on CrossDocked2020 and on Binding MOAD.

The results of the comparison against MolJO are mixed and not very convincing. Nevertheless, I appreciate that the authors no longer claim that their method outperforms MolJO.

In conclusion, I will raise my score to weak accept.

Dear Reviewer kgoQ,

Thank you for the constructive feedback and for raising your score. The revised PDF includes the new results and the clarified MolJO discussion.

Reviewer_HWrDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MOLFRA, an online post-training (alignment) method for pocket-conditioned multimodal molecular generators that produce continuous coordinates and discrete atom types jointly. The starting point is that pretrained SBDD generators are not trained on task-specific properties, while existing property-steering approaches either need sampling-time guidance, gradient-based optimization, or offline preference data, and tend to collapse diversity. MOLFRA instead adapts the forward-process-reconstruction idea (as in DiffusionNFT) to the hybrid flow/jump sampler: fresh rollouts from an EMA anchor are scored with black-box objectives (Vina docking score, synthetic accessibility), and a shared pocket-relative reward weights two paired branches — a coordinate-endpoint branch using self-normalized regression, and an atom-type-logit branch using categorical cross-entropy, the latter constructed so predictions stay on the simplex. No trajectory likelihood ratios and no backpropagation through the sampler are required. The method is instantiated on MOLFORM and evaluated on CrossDocked2020 with Vina Score/Min/Dock, QED, normalized SA, diversity, ligand size, Lipinski criteria, and PoseCheck strain/clash metrics, plus reward-composition and objective ablations over 1,300 rounds.

Critical strengths
  1. The adaptation to a hybrid flow/jump sampler is the genuinely non-trivial part, and it is done cleanly. Forward-process reconstruction methods were developed for purely continuous (or purely discrete) diffusion; SBDD generators mix affine coordinate dynamics with categorical jumps, and the two outputs have different admissible geometries (coordinates extrapolate affinely, atom-type probabilities must stay on the simplex). Splitting the update into a self-normalized endpoint-regression branch and a simplex-preserving logit branch under a shared reward, while leaving the pretrained sampler intact and avoiding both trajectory likelihood ratios and differentiable sampling, is a well-motivated design that a practitioner could actually run at 1,300 online rounds.

  2. The paper is unusually candid about the physical cost of reward optimization, and backs it with the right ablations. Rather than claiming a free lunch, Table 2 reports PoseCheck strain (median 264 vs. 114 for native complexes) and clash counts, and Table 3 abstains from claiming the absence of reward hacking ("rather than establish the absence of reward hacking"). The reward-composition ablation is also informative in a way that goes beyond the headline: adding SA to Vina cuts median strain from 2,139 to 264 and clashes from 4.16 to 2.28 at a small docking cost, which is a useful empirical finding about which proxy rewards act as regularizers.

Critical weaknesses
  1. The optimization target is a docking proxy, and the reported improvements come with a physical-plausibility cost that is larger than the headline suggests. Mean Vina Dock improves from −7.50 to −9.25 kcal/mol, but the same table reports median strain energy of 264 kcal/mol against 114 for the reference complexes, and only 37.1% of poses redock within 2 Å — i.e. the molecules score better while sitting in higher-strain, less physically plausible conformations. Because Vina Dock is precisely the reward being optimized, the improvement is partly definitional. The paper mitigates this by reporting PoseCheck, which is to its credit, but the abstract presents only the docking and SA gains. At minimum the abstract and conclusion should state the strain gap and the redocking rate alongside the gain. Ideally the method should be validated with a reward it did not optimize (e.g. a different docking engine, or an ML affinity predictor) to show the gain is not Vina-specific.

  2. The "fully preserving diversity" claim is close to structural given the objective, and the evidence covers one backbone and one dataset. Diversity is unchanged at 0.78 vs. 0.78 — but the update includes an explicit nonnegative KL penalty relative to the EMA anchor (Section 3.3), so preserving the base distribution is partly what the objective is designed to do; reporting it as an empirical finding overstates what was measured. More importantly, all results come from MOLFORM on CrossDocked2020, with a single reward pair (Vina + SA) as the default. The Limitations acknowledge this, but the contributions list still claims generality ("joint reward alignment for structure-based drug design"). A second backbone (e.g. TargetDiff or a purely continuous model) and a second benchmark would be needed before the method can be recommended as a general post-training recipe.

Questions for authors
  1. Does the Vina Dock improvement transfer to a docking engine that was not used as the reward (e.g. Glide, or an ML-based affinity model)? A single column showing an independent scorer would substantially reduce the reward-hacking concern.

  2. Median strain energy is 264 vs. 114 kcal/mol for native complexes. What fraction of the docking gain survives if poses are filtered by strain (e.g. keeping only molecules below the reference median)? Is there a strain-penalized reward variant you would recommend?

  3. The KL penalty to the EMA anchor makes diversity preservation partly structural. Can you report a diversity metric measured against the initial pretrained checkpoint rather than the EMA anchor, and/or the diversity obtained with the KL term ablated?

  4. How sensitive are the results to the number of online rounds (1,300)? Is there a point at which strain or clash counts begin to degrade faster than docking improves?

  5. Do the reported numbers for the compared methods (MolJO, TAGMol, AliDiff, …) come from re-running their code under your evaluation protocol, or from their papers? If the latter, which metrics are strictly comparable?

6 comments
AuthorsOpenReview ↗

Response to Reviewer HWrD (1/4): additional backbones and a second benchmark (W2)

We thank the reviewer for the careful review, and for recognizing the hybrid flow/jump adaptation and our candor about the physical cost of reward optimization. This comment covers W2 (generality); comment 2 covers W1, Q1 and Q2; comment 3 covers W2 (diversity), Q3 and Q4; comment 4 covers Q5.

Setup. We applied MOLFRA to MolCRAFT (Bayesian flow network) and TargetDiff (DDPM + categorical diffusion) with the same Vina Score + SA reward, pocket-relative normalization (Eq. 2), EMA anchor and objective (Eq. 5). Each backbone keeps its own forward corruption and sampler. TargetDiff's 1,000-step sampler makes rollouts slow, so our new experiments fixed a budget of 20 wall-clock hours on four GB200 GPUs (TargetDiff: 378 rounds; MolCRAFT: 1,428 rounds). On CrossDocked2020, "Pretrained" rows are the published numbers also used in Tables 1-2; "+ MOLFRA" rows and all Binding MOAD rows come from our evaluation pipeline.

CrossDocked2020 (better value in each Pretrained / + MOLFRA pair in bold)

BackboneModelVina Score ↓Vina Min ↓Vina Dock ↓QED ↑SA ↑Diversity ↑Connected ↑Strain (med.) ↓Clashes ↓
MolCRAFTPretrained-6.15-6.99-7.790.480.660.7296.7%1966.91
+ MOLFRA-6.51-7.65-8.390.510.810.7398.9%826.71
TargetDiffPretrained-5.71-6.43-7.410.490.600.7290.4%120810.67
+ MOLFRA-5.84-6.70-7.590.530.620.7194.1%8059.23

Zero-shot transfer to Binding MOAD (130 pockets, 100 molecules each; the CrossDocked2020-trained checkpoints are evaluated directly, without retraining; a tie is not bolded)

BackboneModelVina Score ↓Vina Min ↓Vina Dock ↓QED ↑SA ↑Connected ↑Strain (med.) ↓Clashes ↓
MolCRAFTPretrained-6.50-6.61-6.970.540.7498.5%1236.21
+ MOLFRA-7.42-7.52-7.870.540.8497.7%814.17
TargetDiffPretrained-5.44-6.04-6.980.530.6383.4%75310.86
+ MOLFRA-5.80-6.32-7.220.560.6585.0%6469.60

MOLFRA improves the three Vina metrics, SA, median strain and clash counts on all three backbones and both benchmarks, while unoptimized metrics change little (TargetDiff diversity 0.72 → 0.71; MolCRAFT Connected on MOAD 98.5% → 97.7%). Gains on TargetDiff are smaller (Vina Dock -7.41 → -7.59), likely because only 378 rounds fit in the 20-hour budget, and its median strain stays high (805 vs. 114 for native complexes). We will state the scope in Contributions and Limitations: three backbones and two benchmarks under the Vina + SA reward, with the four-reward ablation on MOLFORM only.

AuthorsOpenReview ↗

Response to Reviewer HWrD (2/4): independent scorers (W1, Q1) and strain-filtered gains (Q2)

Q1: Independent scoring metrics. We rescored the generated ligand poses using GNINA (McNutt et al., 2021) with two scoring metrics not used during the generative model's training or optimization: Vinardo (Quiroga & Villarreal, 2016), an empirical scoring function, and CNNaffinity, a 3D convolutional neural network-based predictor of protein–ligand binding affinity trained on experimental affinity data.

BackboneModelVinardo ↓CNNaffinity ↑
MOLFORMPretrained-4.585.39
+ MOLFRA-5.055.05
MolCRAFTPretrained-5.125.63
+ MOLFRA-5.315.23
TargetDiffPretrained-4.515.44
+ MOLFRA-5.015.54

Vinardo, which re-parameterizes Vina-type terms, improves on all three backbones (by 0.47, 0.19 and 0.50). CNNaffinity improves for TargetDiff (+0.10) but decreases for MOLFORM (-0.34) and MolCRAFT (-0.40). We therefore do not claim that the gain is Vina-agnostic; part of it may be specific to the optimized proxy, which is consistent with your concern.

Manuscript changes (W1). We will add a sentence on the CNNaffinity decreases to Limitations.

Q2: how much of the docking gain survives strain filtering? We kept only molecules with PoseCheck strain energy ≤ 114 kcal/mol (the native median) and compared mean Vina Dock (kcal/mol):

BackboneCheckpointUnfilteredStrain ≤ 114
MolCRAFTPretrained-7.79-6.64
+ MOLFRA-8.39-8.15
Delta (MOLFRA - Pretrained)-0.60-1.51
MOLFORMPretrained-7.50-5.28
+ MOLFRA-9.25-7.23
Delta (MOLFRA - Pretrained)-1.75-1.95

The docking advantage grows under filtering (MolCRAFT -0.60 → -1.51, MOLFORM -1.75 → -1.95), so the gain is not carried only by high-strain poses.

Strain-penalized reward. In our reward ablation, adding SA to Vina already cuts median strain from 2,139 to 264 (-88%) and clashes from 4.16 to 2.28, at a small docking cost (Vina Dock -9.45 → -9.25; Table 5). For stricter control, we recommend adding a clipped force-field strain term (e.g., an MMFF/UFF energy difference before and after local relaxation) to the raw reward; we have not tested this.

AuthorsOpenReview ↗

Response to Reviewer HWrD (3/4): diversity and the KL penalty (W2, Q3) and number of online rounds (Q4)

Q3: diversity and the KL penalty.

Role of KL regularization. We agree that the objective has implicit regularization. DiffusionNFT (Zheng et al., 2026) uses symmetric squared-error branches, which yield a quadratic anchor-relative term; our paired categorical cross-entropy admits an exact decomposition into signed reward feedback and nonnegative anchor-relative KL terms (Eq. (6), Proposition 2). These KL terms arise from the decomposition and are not separately added regularizers. They constrain atom-type predictions relative to a moving EMA anchor, but do not directly constrain coordinates or bound the final molecular distribution's divergence from the pretrained model, so diversity preservation is not guaranteed. We report it as an empirical observation across three backbones: 0.78 → 0.78 (MOLFORM), 0.72 → 0.71 (TargetDiff), 0.72 → 0.73 (MolCRAFT).

Revisions. "Fully preserving" will be replaced by "retaining the pretrained base's diversity (0.78 vs. 0.78)" in the Abstract, Section 4.2 and Conclusion.

Q4: sensitivity to the number of online rounds. We evaluated intermediate EMA checkpoints from the 1,300-round main run (same protocol and test set):

RoundVina Dock ↓SA ↑QED ↑Clash ↓
0 (base)-7.500.600.485.82
300-8.130.640.574.48
600-8.450.690.553.43
900-8.820.730.512.96
1300 (final)-9.250.740.502.28

Clashes fall at every checkpoint (5.82 → 4.48 → 3.43 → 2.96 → 2.28) while Vina Dock and SA keep improving, consistent with the regularizing effect of SA in the reward ablation (Table 5). QED, which is not in the reward, peaks early (0.57 at round 300) and declines to 0.50. The training reward had largely plateaued by round 1,300.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Rebuttal

Dear Reviewers, ACs and SACs,

We thank the reviewers for their careful reading. Our responses address the concerns as follows.

1. Generality. We applied MOLFRA to TargetDiff and MolCRAFT with the same reward and training loop. Both improve in docking, synthetic accessibility, and conformation quality, and both transfer to Binding MOAD dataset without retraining.

2. Reward hacking. The Vina gains do not come at the expense of clash: on both added backbones it decreases together with the Vina scores. We also rescore with GNINA and report that the gains may be partly specific to the Vina reward.

3. Evaluation. We add validity, uniqueness, novelty, and we give the source of every baseline number.

4. Claims. Diversity retention is an empirical observation, not a guarantee. MOLFRA's novelty is its adaptation to hybrid flow/CTMC generators; the forward-process principle comes from DiffusionNFT. We also do not claim state of the art over MolJO.

Best regards,

Submission33439 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.