- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates the multi-objective trade-off between binding affinity and synthetic accessibility in Structure-Based Drug Design (SBDD).
The authors point out that pretrained 3D molecular generation models mainly learn the distribution of training data and do not necessarily satisfy multiple target properties simultaneously. In addition, during reinforcement learning post-training, directly dropping invalid molecules may lead to a decrease in the valid molecule generation rate, while different reward objectives may also compete with each other. To address these issues, the authors adapt the existing Flow-GRPO method to the pretrained 3D molecular generation model FLOWR.ROOT. By penalizing invalid generation trajectories and adjusting the reward weights of different properties, including Vina, SA, and QED, they achieve post-training alignment for multi-objective molecular generation.
The authors conduct post-training and evaluation on the SPINDR dataset and test cross-dataset transfer performance on CrossDocked2020. The experimental results show that adjusting reward weights can change the property preferences of generated molecules. Some configurations can simultaneously improve Vina, SA, and QED while maintaining a high molecular validity rate. In addition, by analyzing changes in molecular descriptors such as rotatable bonds, ring structures, stereocenters, and logP, the authors investigate the statistical correlations between reward configurations and changes in molecular structures and properties.
Critical strengths
S1. The paper identifies and effectively mitigates the decline in molecular validity during reinforcement learning post-training.
The authors compare two approaches: directly dropping invalid generation trajectories and assigning low rewards to them. The experiments show that the latter can better maintain the valid molecule generation rate of the pretrained model while improving the target properties. This reveals an important issue in molecular reward alignment: optimizing properties only among valid generated samples may lead to a decline in overall generation capability.
S2. The paper provides relatively systematic multi-objective reward-weight experiments and cross-dataset validation.
The authors conduct systematic experiments with different Vina–SA reward weights and validate the corresponding trends across multiple training seeds and two datasets, SPINDR and CrossDocked2020. The experiments show that different reward configurations can produce different molecular property preferences, and some configurations can simultaneously improve multiple objectives relative to the pretrained baseline. In addition, the authors analyze changes in the structural and physicochemical characteristics of generated molecules, providing some empirical evidence for understanding the behavior of reward alignment.
Critical weaknesses
W1. The methodological novelty is relatively limited, and the paper does not sufficiently demonstrate substantive differences from existing multi-objective optimization methods.
The core framework of this paper is mainly built upon the existing FLOWR.ROOT and Flow-GRPO methods, while multi-objective property control is primarily achieved by adjusting linearly weighted rewards. Although this combination has practical value, the paper lacks a new multi-objective optimization algorithm, an adaptive reward control strategy, or other clearly distinctive methodological designs.
Therefore, the main contribution of the current work is closer to an application and empirical study of existing reinforcement learning methods in 3D molecular generation, rather than the proposal of a new multi-objective control mechanism.
Although the authors provide comparison results with methods such as MolJO, MolForm-DPO, and PILOT in the appendix, these methods differ in training data, generative backbones, and optimization objectives. Therefore, these comparisons are insufficient to demonstrate the advantages of Flow-GRPO itself.
A more convincing validation would compare other reasonable post-training or reward optimization strategies, such as reward-weighted fine-tuning, rejection sampling, or other applicable preference alignment methods, under the same pretrained backbone, training data, and comparable computational budgets. Such experiments would help clarify the unique value and necessity of the current framework compared with existing methods.
W2. The mechanistic analysis of molecular structures and property trade-offs mainly remains at the level of statistical correlations.
In Figure 4, Figure 5, and the appendix, the authors analyze the relationships between molecular descriptors, including rotatable bonds, ring structures, stereocenters, and logP, and improvements in Vina, SA, and QED.
These analyses can demonstrate certain statistical correlations between reward alignment and molecular structural changes. However, they lack targeted intervention experiments or mechanistic ablations, making it difficult to further verify which structural changes actually drive the trade-offs between different objectives.
In addition, some observed relationships are consistent with the existing preferences of scoring functions such as Vina and SA for molecular flexibility or structural complexity. Therefore, the extent to which these statistical results constitute new scientific findings still requires further discussion.
If the authors could use more targeted controlled experiments to distinguish the effects of molecular descriptors from the direct preferences of the scoring functions, it would significantly strengthen the persuasiveness of their mechanistic analysis.
Questions for authors
Q1. Has the Trade-off Frontier in Figure 3 been sufficiently validated?
Figure 3 presents an empirical trade-off curve by connecting the average Vina–SA results corresponding to different reward weights. However, some reward configurations outperform others in both reported average metrics. For example, on SPINDR, the configuration with w_Vina = 0.25 achieves better average Vina and SA scores than the configuration with w_Vina = 0. This means that, based on the reported mean values, not all points lie on a Pareto-optimal frontier in the strict sense.
Could the authors further analyze the Pareto efficiency of different reward configurations while considering the uncertainty introduced by random seeds? In addition, can these results support a stable and predictable relationship between reward weights and final molecular properties, rather than merely providing several empirical operating points obtained by training with different weights?
Q2. How do the authors explain the deterioration of some molecular geometric distribution metrics after reward alignment?
Although some reward configurations can simultaneously improve Vina, SA, and QED, Table 1 and Appendix Table 5 show that the Wasserstein distances of bond-angle and bond-length distributions deteriorate to varying degrees after post-training compared with the pretrained model.
This suggests that property reward optimization may also impair some of the original generative model's ability to match geometric distributions, while this additional trade-off has not been sufficiently discussed.
Could the authors further explain the sources of these distributional changes and their relationship with PoseBusters validity? In particular, do these observations suggest that the current Vina–SA trade-off analysis does not fully reflect the effects of reward alignment on overall molecular quality?
Q3. Could the probability calculation and normalization of the mixed continuous-discrete policy be further clarified?
Appendix B.2 normalizes the log-likelihoods of different molecular modalities by the number of atoms or valid atom pairs to reduce the influence of molecular size on policy updates. However, the normalized likelihood ratio is not exactly equivalent to the importance ratio of the original joint transition distribution.
Could the authors further explain the influence of this modification on the GRPO optimization objective and the rationale for choosing the current normalization approach?
In addition, the stochastic coordinate sampling in Appendix A becomes deterministic when t >= 0.9, while Appendix B still uses Gaussian transition likelihood and KL formulas. Could the authors clarify how zero-variance coordinate transitions are handled at this stage in the actual implementation to ensure the numerical stability of the probability calculations?