- Rating
- 2
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes GRAB, a framework for fine-tuning molecular Bayesian flow networks using group-relative policy optimization. It formulates MolCRAFT’s sampled coordinate belief means and atom-type logits as joint actions and optimizes their likelihoods using terminal molecular rewards.
Critical strengths
The policy formulation explicitly accounts for the stochastic variables sampled by the generator. The framework accommodates non-differentiable molecular rewards without requiring property-predictor gradients during inference. Reported improvements in docking and molecular quality demonstrate practical potential.
Critical weaknesses
Limited methodological novelty. CByG already investigates property-driven control of molecular BFN belief updates, BFN-DPO establishes multi-reward BFN alignment, and FTDiff applies trajectory-level GRPO to diffusion-based SBDD. Against these precedents, GRAB’s novelty largely reduces to defining actions and Gaussian likelihoods for a particular BFN sampler. The manuscript does not sufficiently establish why this adaptation constitutes a substantial methodological advance.
Insufficient isolation of the proposed contributions. The experiments do not establish the individual benefits of joint-action optimization, log-density scaling, and grouping choices over a straightforward policy-gradient implementation.
Limited comparative evidence. The BFN-DPO comparison is not matched in candidate-generation budget, complicating attribution of improvements to the optimization method. Single-run results and evaluation on one backbone further limit conclusions about robustness and generality.
Questions for authors
What nontrivial methodological obstacle does GRAB resolve beyond applying standard GRPO to the known Gaussian sampling distributions? Please clarify its distinction from CByG, BFN-DPO, and FTDiff.
Which existing results establish the necessity of joint-action optimization, log-density scaling, and the proposed grouping strategy?
Please clarify the total generation and oracle-evaluation budgets for GRAB and BFN-DPO, and whether the reported advantage remains supported under comparable budgets.
7 comments
Response to Reviewer f8HR (W2/Q2 (1))
W2/Q2: Joint-Action Optimization, Log-Density Scaling, and Grouping
1. Joint-action formulation
By joint-action optimization, we mean using terminal molecular feedback to update the policy for the complete sampled belief update. At each step, the action consists of the sampled coordinate-belief means and sampled type logits, with giving the next atom-type belief. Both action distributions depend on the shared coordinate and atom-type belief state [4]. The term joint refers to accounting for both components of this sampled action using the same reward-derived advantage.
Conditioned on the current state, pocket, and atom count, the joint action density factorizes into coordinate and type distributions. With the sampled state and action held fixed, its log-density gradient therefore contains both contributions. Omitting either term would generally omit part of this gradient. Computing the two log-density terms separately and summing them is an equivalent way to evaluate the joint policy.
2. Log-density scaling
We use log-density scaling to reduce the risk that accumulated changes across a high-dimensional joint action prematurely truncate reward-driven updates through PPO clipping. For one molecule at one sampling step, the joint log-density sums over scalar components. Consequently, changes across individual components can accumulate and push the joint likelihood ratio beyond the clipping interval. We therefore use
\Delta=\log\pi_\theta(a\mid s)-\log\pi_{\mathrm{old}}(a\mid s),$$ with $D=24(3+13)=384$, where 24 is the approximate mean number of heavy atoms. This matters because PPO clipping directly affects whether a sampled action continues to contribute to learning [5]. Here, $A$ denotes the group-relative advantage. When $A>0$ and $r_D>1+\epsilon$, or $A<0$ and $r_D<1-\epsilon$, the selected surrogate branch is constant with respect to the ratio, and that sample contributes zero policy-surrogate gradient. Scaling brings the ratio closer to one for a given joint log-density change, reducing the risk of reaching these flat regions during repeated updates on a rollout batch. We compared $D=384$ and $D=1$ over the first 150 training iterations: | Ratio interval | Iterations containing out-of-range ratios: $D=384$ | $D=1$ | |---|---:|---:| | $[0.99,1.01]$, the training clipping interval | 1/150 | 150/150 | | $[0.8,1.2]$, a broader diagnostic interval | 0/150 | 38/150 | Thus, without scaling, out-of-range ratios occurred in every observed iteration under the configured clipping interval, and remained present in 38 iterations even when assessed against $[0.8,1.2]$. Scaling substantially reduced their occurrence. These iteration-level counts measure the occurrence of out-of-range ratios; they do not quantify the fraction of samples with zero surrogate gradient. To assess molecular performance, we also compared D=384 with D=1 using the same initialization, reward weights, training budget, and learning rate, changing only the log-density scaling factor. Vina entries report **mean / median**, and SE entries report the **25th / 50th / 75th percentiles**. Vina scores and SE are in kcal/mol. | Setting | Vina Score ↓ | Vina Min ↓ | Vina Dock ↓ | SE ↓ | SA ↑ | QED ↑ | Div ↑ | |---|---:|---:|---:|---:|---:|---:|---:| | D=1 | −5.75 / −6.83 | −7.01 / −7.19 | −8.02 / −8.21 | 61 / 162 / 318 | 0.75 | 0.52 | 0.76 | | GRAB-B, D=384 | −7.593 / −8.428 | −8.759 / −8.736 | −9.515 / −9.535 | 44 / 104 / 200 | 0.76 | 0.51 | 0.75 | D=384 achieves better results across all Vina metrics and SE quartiles. Mean Vina Dock improves from −8.02 to −9.515, and median SE decreases from 162 to 104. SA also increases from 0.75 to 0.76, while D=1 retains slightly higher QED and diversity, both by 0.01. Together with the ratio statistics above, this ablation supports the practical benefit of log-density scaling: under the same training settings, it reduces the occurrence of out-of-range ratios and improves docking performance and conformational strain. We have added the ratio statistics and complete molecular results to Appendix J, referenced from Section 4.2.Response to Reviewer f8HR (W2/Q2 (2))
3. Grouping and comparison with a straightforward policy-gradient implementation
We form each group from molecules generated for the same pocket and with the same sampled atom count. Reward distributions can vary across pockets and ligand sizes, including across different atom counts within the same pocket. Grouping by both pocket and atom count allows us to compare candidates under the same conditions. Normalizing each reward channel within these groups then makes the advantage reflect relative molecular performance, reducing the influence of differences in reward levels and variability across pockets and ligand sizes.
We agree that a straightforward policy-gradient implementation is an important comparison. To our knowledge, a published REINFORCE baseline for the same MolCRAFT setting is not available. Following your suggestion, we therefore implemented REINFORCE [6] on the MolCRAFT backbone and compared it with GRAB-B using the same training data and reward weights: (0.35, 0.15, 0.15, 0.35) for Vina Min, QED, SA, and SE, respectively.
| Method | Vina Score Avg ↓ | Med ↓ | Vina Min Avg ↓ | Med ↓ | Vina Dock Avg ↓ | Med ↓ | SE 25% ↓ | 50% ↓ | 75% ↓ | SA ↑ | QED ↑ | Div ↑ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MolCRAFT | −6.307 | −6.842 | −7.089 | −7.106 | −7.808 | −7.907 | 66 | 153 | 346 | 0.69 | 0.53 | 0.74 |
| REINFORCE (our implementation) | −6.546 | −7.143 | −7.381 | −7.365 | −8.116 | −8.189 | 75 | 171 | 414 | 0.69 | 0.55 | 0.72 |
| GRAB-B | −7.593 | −8.428 | −8.759 | −8.736 | −9.515 | −9.535 | 44 | 104 | 200 | 0.76 | 0.51 | 0.75 |
REINFORCE improves docking scores over the initial MolCRAFT model, but its strain quartiles increase and diversity decreases. GRAB-B achieves substantially better docking scores and lower strain than REINFORCE: mean Vina Dock improves from −8.116 to −9.515, and median SE decreases from 171 to 104. SA also increases from 0.69 to 0.76, and diversity from 0.72 to 0.75. REINFORCE achieves higher QED (0.55 versus 0.51).
Under the same reward weights and training data, the complete GRAB procedure with group-relative updates achieves stronger docking performance, lower strain, and higher SA and diversity than REINFORCE. This comparison supports the effectiveness of the complete procedure, although it does not establish the necessity of grouping alone. We thank the reviewer for suggesting this comparison. We have clarified the grouping rationale in Sec. 4.2 and added the REINFORCE comparison setting and complete results in Appendix H, with a summary in Sec. 5.2.
Response to Reviewer f8HR (W1/Q1)
W1/Q1: BFN-specific Contribution and Distinction from Prior Work
Thank you for asking us to clarify the methodological contribution. GRAB’s methodological contribution is an on-policy formulation that makes the joint belief-update process of a molecular BFN trainable from black-box molecular evaluations. This enables the generator to improve through repeated generation–evaluation–update cycles, while retaining its original sampling recursion and requiring no additional property-gradient guidance at inference.
1. BFN-specific formulation
Knowing the Gaussian sampling distributions does not by itself specify the appropriate policy likelihood. In a molecular BFN, the network predicts clean coordinates and atom-type probabilities, but the sampler draws coordinate-belief means and type logits. The terminal reward evaluates a reconstructed molecule downstream of these decisions. GRAB connects these levels by defining the sampled pair (ai=(μi,yi)) as the joint action and placing softmax in the deterministic state transition. We derive the gradients of the joint action log-density for both coordinate and atom-type predictions, allowing terminal molecular scores to guide updates to both branches through the recorded sampling trajectories (Sec. 4.1; App. A.1–A.2). This is the methodological role of the formulation: it specifies how the existing BFN sampling dynamics can support reward-driven policy learning.
2. Distinction from prior work
CByG [1] uses property-predictor gradients to control sampling with a fixed generator. GRAB instead updates generator parameters from on-policy terminal evaluations and subsequently generates without additional property-gradient guidance.
BFN-DPO [2] learns from reward-derived preference pairs through a reference-relative objective based on BFN sender/receiver distributions. GRAB uses the actions actually sampled in fresh rollouts and their group-relative advantages for policy updates.
FTDiff [3] already applies trajectory-level GRPO and explicitly formulates a joint coordinate/type likelihood ratio for diffusion. GRAB defines its actions over BFN belief variables. Its practical objective also clips a scaled joint action ratio, whereas FTDiff separately clips coordinate/type terms. This difference alone does not establish empirical superiority.
Our contribution is the explicit BFN policy formulation and the resulting training procedure. The score-function derivation concerns an objective with a frozen terminal output rule, and the implemented clipped objective is a surrogate (App. A.2–A.4). The comparisons with other methods evaluate the complete procedure; the additional D=1 comparison in W2/Q2 examines log-density scaling. We have clarified the distinction from prior work in Sec. 2 and the role of the joint-action formulation in Sec. 4.1.