- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces MOLFRA, an online post-training method for pocket-conditioned multimodal molecular flows. It uses forward-process reconstruction with paired positive and negative branches to jointly align coordinate and atom-type generation under shared pocket-relative rewards. Experiments are mainly conducted on CrossDocked2020 with MOLFORM as the backbone, mainly using SA and vina-based docking score for evaluations.
Critical strengths
The proposed method is generally clear and easy to follow. The overall framework, training procedure, and the roles of different components are explained in a relatively straightforward manner, making the main idea easy to understand.
Critical weaknesses
-
The methodological novelty may be limited. This work appears to be largely established upon the previous work, such as its core learning rationales and continuous coordinate construction, and then performs positive/negative branching for atom types. It is unclear if there are any substantial new rationales, frameworks, theories, or insights introduced.
-
Several major concerns for experiments. (1) The key performance metrics overlap with the training reward (e.g. vina score & SA), which may have risks of reward hacking. A good vina score while extremely high strain energy, and abnormal ring-size distributions may further imply this concern, potentially meaning obviously abnormal structures in generated molecules. (2) The evaluation protocol may have severe bias. Each pocket only generates 100 molecules, which may be insufficient. Most of the key metrics are computed based on valid or successfully evaluated molecules, while the overall validity, novelty, uniqueness of generated molecules are not reported. It is possible that there are lots of failed samples but only a few valid molecules with good vina score. Given this, the claim that “fully preserving diversity” may be not supported as well. (3) Baseline comparison may be unfair. In related work, several trajectory-based online RL methods are discussed. However, there is no standard online RL method under the same setting included for comparison. In addition, the results reported in table 1 seem to be cross-paper. The experimental settings may be quite different and not comparable. (4) Experimental scope and reliability are quite limited. The experiments are mainly conducted on just one dataset and backbone. There are also no multi-seed tests or other statistical tests. The optimization tasks may have a relatively high degree of randomness so it is unclear whether the improvements are reproducible.
-
Potential inconsistency between methods and actual training. (1) Reward-surrogate alignment appears to be established under a fixed rollout-corruption law, posterior calibration, and trainable-anchor agreement. However, the model may keep updating and the EMA anchor may also keep changing during actual training. The assumptions or premises may not always hold during the training. (2) It is unclear if this is an RL method. Although the title includes “RL”, there are no on-policy rollouts from the current model as well as trajectory likelihood ratios, policy gradients, and backpropagation via sampling trajectory, which are commonly used components in the RL framing.
Questions for authors
-
What are the actual novelties and contributions?
-
Are there more convincing experimental results that support the claims?
-
If the premises do not always hold, can the method still work as intended? And could the method be called “RL”?
8 comments
Response to Reviewer QMYc (2/4): additional backbones and a second benchmark (W2.4, Q2)
We added two pretrained backbones and a second test set. MOLFRA only needs clean-coordinate estimates and atom-type logits, so it transfers across generative paradigms without changing the reward (Vina Score + SA), the pocket-relative normalization (Eq. 2) or the EMA training loop. We apply it to TargetDiff (DDPM with categorical diffusion) and MolCRAFT (Bayesian flow network), each with its own forward corruption and sampler (Section 4.3, Appendix H).
CrossDocked2020 (Table 3; the better value in each pair is in bold):
| Backbone | Model | Vina Score ↓ | Vina Min ↓ | Vina Dock ↓ | QED ↑ | SA ↑ | Diversity ↑ | Connected ↑ | Strain (med.) ↓ | Clashes ↓ |
|---|---|---|---|---|---|---|---|---|---|---|
| MolCRAFT | Pretrained | -6.15 | -6.99 | -7.79 | 0.48 | 0.66 | 0.72 | 96.7% | 196 | 6.91 |
| + MOLFRA | -6.51 | -7.65 | -8.39 | 0.51 | 0.81 | 0.73 | 98.9% | 82 | 6.71 | |
| TargetDiff | Pretrained | -5.71 | -6.43 | -7.41 | 0.49 | 0.60 | 0.72 | 90.4% | 1208 | 10.67 |
| + MOLFRA | -5.84 | -6.70 | -7.59 | 0.53 | 0.62 | 0.71 | 94.1% | 805 | 9.23 |
Zero-shot transfer to Binding MOAD (Table 4; 130 pockets from the DiffSBDD split, 100 molecules each; Vina columns are medians over successfully evaluated molecules). We evaluate the CrossDocked2020-trained checkpoints without retraining. The better value in each pair is in bold, and ties are not bolded.
| Backbone | Model | Vina Score ↓ | Vina Min ↓ | Vina Dock ↓ | QED ↑ | SA ↑ | Connected ↑ | Strain (med.) ↓ | Clashes ↓ |
|---|---|---|---|---|---|---|---|---|---|
| MolCRAFT | Pretrained | -6.50 | -6.61 | -6.97 | 0.54 | 0.74 | 98.5% | 123 | 6.21 |
| + MOLFRA | -7.42 | -7.52 | -7.87 | 0.54 | 0.84 | 97.7% | 81 | 4.17 | |
| TargetDiff | Pretrained | -5.44 | -6.04 | -6.98 | 0.53 | 0.63 | 83.4% | 753 | 10.86 |
| + MOLFRA | -5.80 | -6.32 | -7.22 | 0.56 | 0.65 | 85.0% | 646 | 9.60 |
The three Vina metrics, SA, median strain and clash count improve on both backbones and both test sets, so the gains are not specific to MOLFORM or to CrossDocked2020.
TargetDiff's sampler needs 1,000 steps, against 100 for MOLFORM and MolCRAFT, so its rollouts take much longer. We therefore gave both new backbones the same wall-clock budget of 20 hours on 4 GB200 GPUs instead of matching the number of rounds: 378 for TargetDiff and 1,428 for MolCRAFT. The pretrained CrossDocked2020 rows repeat the TargetDiff and MolCRAFT rows of Tables 1-2.
On multiple seeds: we did not run multi-seed replicates. The same recipe improves three backbones on CrossDocked2020 and two on Binding MOAD, which makes a seed-specific artifact less likely but does not replace replicates.
Response to Reviewer QMYc (3/4): reward hacking, independent scorers, strain filtering and ring sizes (W2.1, Q2)
1. Independent scorers (GNINA Vinardo and CNNaffinity). To test whether the gains carry over to scorers outside the reward, we rescored the generated poses with GNINA, using Vinardo (an empirical scoring function) and CNNaffinity (a 3D CNN affinity predictor trained on experimental binding data):
| Backbone | Model | Vinardo ↓ | CNNaffinity ↑ |
|---|---|---|---|
| MOLFORM | Pretrained | -4.58 | 5.39 |
| + MOLFRA | -5.05 | 5.05 | |
| MolCRAFT | Pretrained | -5.12 | 5.63 |
| + MOLFRA | -5.31 | 5.23 | |
| TargetDiff | Pretrained | -4.51 | 5.44 |
| + MOLFRA | -5.01 | 5.54 |
The result is mixed. Vinardo improves on all three backbones. CNNaffinity improves on TargetDiff (+0.10) and drops on MOLFORM (-0.34) and MolCRAFT (-0.40). We therefore do not claim the gains are independent of the scorer, and the Limitations section now notes that the gains may be specific to the reward.
2. Are the docking gains driven by high-strain conformations? We tested this two ways.
Strain-filtered docking. We kept only molecules whose PoseCheck strain energy is at most 114 kcal/mol (the median of the experimental reference complexes) and compared mean Vina Dock (kcal/mol):
| Backbone | Checkpoint | Unfiltered Vina Dock ↓ | Strain Vina Dock ↓ |
|---|---|---|---|
| MolCRAFT | Pretrained | -7.79 | -6.64 |
| + MOLFRA | -8.39 | -8.15 | |
| (MOLFRA Pretrained) | -0.60 | -1.51 | |
| MOLFORM | Pretrained | -7.50 | -5.28 |
| + MOLFRA | -9.25 | -7.23 | |
| (MOLFRA Pretrained) | -1.75 | -1.95 |
The docking advantage over the pretrained base survives the filter, and the margin grows (-0.60 to -1.51 on MolCRAFT, -1.75 to -1.95 on MOLFORM).
Strain and clashes. MOLFRA lowers median strain from 196 to 82.4 on MolCRAFT (native reference: 114) and from 1208 to 804.9 on TargetDiff (Table 3). On MOLFORM, adding SA to the Vina reward cuts median strain by 88%, from 2,139 to 264 (Table 5), and the clash count of 2.28 is the lowest in Table 2.
3. Ring sizes (Appendix G). In Table 7, MOLFRA produces no 3-membered rings (0.0%) and few 4-membered rings (0.6%), and its share of 6-membered rings (68.5%) is close to the reference (66.0%). It does shift mass from 5-membered rings (18.5% vs. 29.6% in the reference) toward 7-membered rings (9.2% vs. 0.8%). Other 3D generators show the same shift (TargetDiff 11.7%, DecompDiff 11.9%, TAGMol 15.0% for 7-rings), and Appendix G already reports it.
Response to Reviewer QMYc (1/4): novelty (W1, Q1), theoretical premises (W3.1, Q3), and the RL framing (W3.2, Q3)
We thank the Reviewer QMYc for the careful review. W1, W3, Q1 and Q3 are answered here; W2 and Q2 are answered in Comments 2 to 4.
W1 & Q1: What are the actual novelties and contributions?
The continuous coordinate branching and the self-normalized loss follow DiffusionNFT, and we do not claim them as new. Our contribution is extending forward-process reward alignment to SBDD generators that couple continuous coordinates with discrete atom types.
We therefore do not present a new forward-process RL principle. The contribution is its simplex-preserving multimodal extension, the analysis of categorical feedback, and evidence across different SBDD generators.
W3(1) & Q3: Do the premises hold during training, and can the method still work as intended?
The premises are sufficient for our local theoretical analysis, not prerequisites for applying MOLFRA. We distinguish two levels of results:
In practice, we use a small learning rate (5×10−7), gradient clipping (8.0) and an EMA anchor (τ=0.995) to promote stability. Empirical effectiveness is supported independently of the theory by the improvements on three backbones on CrossDocked2020 and the transfer to Binding MOAD (Comment 2).
W3(2) & Q3: Can the method be called "RL"?
Yes, in the sense of DiffusionNFT: online, reward-driven policy optimization. Each round, fresh rollouts from the EMA policy πθold are scored with black-box rewards (Vina and SA), and the generator is updated through a reward-dependent forward-process reconstruction objective. Unlike conventional policy-gradient RL, MOLFRA requires neither trajectory likelihood ratios nor backpropagation through sampling (Sec. 3.3). It is also not strictly on-policy with respect to πθ: samples come from the lagged EMA policy (Sec. 3.2), which acts as the behavior policy in each round of rollout and update. We will clarify this distinction at the first mention of "RL."