acceptodds
Under review as a conference paper at ICLR 2027

Reward Design for Post-Training Molecular Generators

Abstract

Diffusion and flow-matching models can generate 3D molecules conditioned on protein pockets, but their training objectives do not directly optimize molecular properties relevant to practitioners. Post-training can shift the generative distribution toward molecules with better properties, yet its success depends on how rewards are constructed: molecular properties may be undefined for chemically invalid outputs, and optimizing one objective can degrade geometric plausibility or other desirable properties. We investigate these challenges by adapting DiffusionNFT, a recently proposed method for negative-aware finetuning, to jointly align the continuous coordinates and categorical outputs of a pocket-conditioned molecular generator. We first show that excluding chemically invalid generated samples from the alignment objective can cause validity collapse, whereas incorporating a validity reward largely prevents this failure. Building on this validity-aware formulation, we present the FlowR.NFT model family to systematically investigate combinations of chemical validity, Vina, QED, SA, and PoseBusters geometry rewards and their respective trade-offs. We identify reward compositions that improve Vina affinity, synthetic accessibility, and drug-likeness over the baseline model, with FlowR.NFT achieving a redocked Vina score of on SPINDR while largely preserving molecular validity. These property gains transfer well to testing on CrossDocked2020, with a Vina docking score of , better than those of all prior baselines in our benchmark. However, the plausibility of generated ligand poses is less well preserved across datasets. Together, our results highlight the importance of careful invalid-sample handling and balancing molecular properties during fine-tuning of ligand generators.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
structure-based drug design, 3D molecular generation, flow-matching, online reward alignment, molecular property optimisation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1ZaSOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces the FlowR.NFT, which incorporates DiffusionNFT method to optimize Flowr.root model with binding affinity, drug-likeness, and synthetic-accessibility score. The major objective of this work is addressing the validity collapse problem and find the reward compositions for Flowr.root finetuning. In experiments, authors evaluate the proposed method on various reward compositions and compare it with various diffusion and flow-based methods.

Critical strengths

None

Critical weaknesses
  1. The primary motivation of this paper is the validity collapse during optimization. However, it is not validated that this validity collapse issue is raised not only Flowr.root but also various diffusion-based methods. The authors' experiments on Flowr.root is the sole evidence of this claim. In this paper, there is no additional reference which addresses or measures the validity collapse issue of diffusion-based sbdd methods. In particular, in Table 1, MolJo and AliDiff show higher validity than Flowr.NFT+Vina. This indicates that they do not suffer the same issue which author observed in Flowr.root.
  2. The raw Vina scores are poor. The main rationale for developing 3D pocket-based molecule generators is the understanding that protein-ligand interactions are crucial for target-aware drug design—specifically regarding target specificity. If the generator does not aim to produce feasible binding conformations, then 3DSynthFlow [1]—which jointly generate 2D molecular graph with its 3D binding pose —demonstrates superior performance compared to most baselines reported in this paper (Vina: -9.42 kcal/mol; QED: 0.73; AiZynthFinder retrosynthesis success rate: 36%). Furthermore, compared to MolJO, the difference in redocked Vina scores is smaller than the reported standard deviation (-9.10 ± 0.09 kcal/mol vs. -9.07 kcal/mol), and the QED score is slightly lower (0.52 vs. 0.55). Considering the trade-off between QED and Vina docking scores, it is difficult to claim that the proposed method is superior to MolJO.

References:

  1. Shen, Tony, et al. "Compositional flows for 3D molecule and synthesis pathway co-design." ICML 2025.
Questions for authors
  1. Is this issue of 'validity collapse' a phenomenon commonly observed across various diffusion models? If this is a general problem, can the proposed method resolve it for other models?
  2. Do the trained FlowR.NFT models retain the target specificity? This can be evaluated via Delta score [1]
  3. Please explain the reason why the Vina score of FlowR.NFT is poor compared to other baselines?
  4. Please report the protein-ligand interaction profile using PoseCheck [2] -- the number of H-Bond and Van der Waals interaction -- and other drug-like properties, such as molecular weight, number of heavy atoms, number of rings, and number of aromatic rings, and compare the metric with baseline models. It is important since it is easy to hack the docking score by increasing the number of atoms or hydrophobic groups [3].

References:

  1. Gao, Bowen, et al. "Rethinking specificity in sbdd: Leveraging delta score and energy-guided diffusion." ICML 2024.
  2. Harris, Charles, et al. "Posecheck: Generative models for 3d structure-based drug design produce unrealistic poses." (2023).
  3. Qu, Yanru, et al. "MolCRAFT: structure-based drug design in continuous parameter space." ICML 2024.
5 comments

We thank the reviewer for the detailed review. Our responses below include new experimental results, clarifying the findings and limitations of our approach.

Scope of Validity collapse: Our experiments establish validity collapse for Flowr.root under valid-only online NFT fine-tuning. Our experiments do not establish how common this issue is across molecular generators or alignment methods. It is, however, a common issue across online post-training methods for molecular generators that molecular rewards can be undefined on chemically invalid samples. Thus, any online fine-tuning method that encounters chemically invalid outputs must specify how to handle these undefined rewards. In contrast, MolJO uses inference-time guidance, and AliDiff uses offline preference alignment. Their validity scores therefore do not test the online fine-tuning failure mode studied here. We will clarify this scope and identify evaluation across additional generators and online alignment algorithms as future work.

Poor score-only VINA scores on Crossdocked: Our original reward uses redocked Vina, which evaluates affinity after an external pose search and therefore does not directly reward the initial ligand placement. While our models fine-tuned on SPINDR achieve good Vina score-only results and intermolecular PoseBusters validity remains at least 0.90 on the SPINDR test set (Table 3), performance on zero-shot transfer to CrossDocked might be limited by this choice.

To address this, we additionally trained FlowR.NFT using score-only Vina, which evaluates the generated pose directly instead of redocked Vina. For FlowR.NFT_multi, this improves score-only Vina from −1.14 to −4.49 kcal/mol on CrossDocked2020, intermolecular PB validity from 0.58 to 0.80, and overall PB validity from 0.56 to 0.73. However, we want to emphasize that when comparing these new results and the results presented in Table 3 to the baseline models in Table 3, one has to take into account that, unlike our FlowR.NFT, all the baselines have been finetuned on CrossDocked2020. The Vina+QED+Geom configuration achieves a stronger pose-quality trade-off on CrossDocked2020: score-only Vina reaches −7.15 kcal/mol, while intermolecular validity reaches 0.9 and redocked Vina reaches −9.10 kcal/mol, although SA is lower at 0.53.

On SPINDR, score-only Vina of FlowR.NFT_multi improves from −7.43 to −8.73 kcal/mol and intermolecular validity from 0.90 to 0.98 when using Vina score-only in the reward.

Thus, direct pose feedback can mitigate the pose deterioration on zero-shot CrossDocked transfer. We will include the full comparisons across reward configurations in the revised manuscript.

Comparison to previous methods: We thank the reviewer for highlighting 3DSynthFlow. While its reported −9.42 kcal/mol is Vina docked, rather than score-only Vina, this strong docking, QED and retrosynthesis results make it a relevant comparison, which we will discuss in the manuscript. A direct numerical ranking would require evaluation under a common protocol. We also agree that the small difference in mean redocked Vina does not establish superiority over MolJO. We will describe the original results as competitive docking performance with different molecular-property trade-offs. This comparison also involves different training distributions: FlowR.NFT is pretrained and fine-tuned only on SPINDR and evaluated on CrossDocked2020 without further training or checkpoint selection, whereas MolJO and 3DSynthFlow use CrossDocked2020 training data. Our results therefore demonstrate transfer across datasets, although this difference prevents us from attributing performance differences solely to the alignment method.

Potential exploitation of the Vina reward: We agree that docking improvements should be assessed alongside molecular composition and protein–ligand interactions. Our sampling protocol retains the same atom-count prior, and the observed heavy-atom distributions remain unchanged after fine-tuning. However, optimisation with Vina reward increases ring counts, and PoseCheck results show fewer hydrogen-bond interactions than the pretrained model, while van der Waals contact counts remain comparable or higher (using redocked poses for docked-reward model due to poor original-pose validity). Further, FlowR.NFT-multi with score-only Vina reward shows higher hydrogen-bond and VdW contact counts and lower aromatic-ring counts.

FlowR.NFT optimises the supplied reward and therefore inherits its limitations. The framework can accommodate alternative affinity estimators without requiring differentiable rewards. Evaluating whether estimators better correlated with experimental affinity reduce these biases is an important direction for future work. We will include the descriptor and interaction comparisons in the manuscript.

ModelPC-HBAcceptorPC-HBDonorPC-VdWContactHeavy atomsRingsAromatic rings
MolJO0.540.194.4122.923.882.26
Flowr.root1.110.415.4122.602.731.69
FlowR.NFT-multi (docked Vina)0.20 ± 0.020.08 ± 0.005.03 ± 0.1222.61 ± 0.004.88 ± 0.222.30 ± 0.30
FlowR.NFT-multi (score-only Vina)0.37 ± 0.020.19 ± 0.006.31 ± 0.0122.60 ± 0.014.78 ± 0.150.97 ± 0.16

Thank you for the clarifications and additional experiments. I still have the following concerns.

  1. Trade-offs relative to pretrained Flowr.root. The score-only-trained FlowR.NFT-multi improves upon the original fine-tuned model, but remains worse than pretrained Flowr.root on CrossDocked in raw Vina (−4.49 versus −5.96), intermolecular PB validity (0.80 versus 0.93), and overall PB validity (0.73 versus 0.91). Thus, even the directly optimized score-only objective does not improve over the pretrained baseline in this transfer setting. The Vina+QED+Geom configuration improves Vina to −7.15, but lowers SA from 0.71 to 0.53. Please provide the complete results, including variability, to clarify what is gained and lost through fine-tuning. Fewer H-Bond and more VdW interactions relative to the pretrained model also raise concerns about altered interaction patterns and potential false positives [1].

  2. Target specificity. My question remains unanswered. Please provide Delta scores or an equivalent intended-target/off-target evaluation.

  3. Clarification regarding 3DSynthFlow. I introduced this method to highlight the importance of directly generated pose quality, not to request an additional baseline. Improving molecular proposals for subsequent docking differs from improving directly generated binding poses.

References:

  1. Qu, Yanru, et al. "MolCRAFT: structure-based drug design in continuous parameter space." ICML 2024
Reviewer_ik3BOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies online post-training for pocket-conditioned molecular generators, focusing on how chemically invalid outputs should participate in learning when their property rewards cannot be computed. The authors extend DiffusionNFT to continuous coordinates and discrete molecular features. They retain invalid samples, assign neutral rewards to undefined properties, and provide negative feedback through chemical validity. They then study different combinations of docking, QED, synthetic accessibility, and geometry rewards.

Critical strengths

In online generation, which outputs can be scored depends on what the model currently generates. Excluding unscorable outputs changes the feedback available for subsequent updates. An improvement in average property scores among scorable molecules can therefore coexist with deterioration of the overall generative distribution.

Equation 8 and Appendix A.1 clearly distinguish sample exclusion, neutral feedback, and negative feedback. Under the NFT objective used here, a reward of 1/2 removes the directional update from that property while retaining regularization relative to the current rollout policy. The formulation separately expresses the absence of a property preference when a property is undefined and negative feedback when chemical invalidity is known.

Critical weaknesses

The introduction asks how to improve molecular properties while maintaining validity and balancing geometric requirements. The paper explicitly analyzes how undefined rewards enter the NFT update, while the multi-objective section mainly presents reward configurations and their outcomes. In Equation 8, the coefficient of drift regularization remains fixed, while reward composition changes the directional update term. Appendix A.1 also explains that introducing a neutral component can attenuate the directional effect of existing rewards. Changing reward composition can therefore alter both objective preferences and the strength of the update relative to regularization toward the current rollout policy. The main text largely interprets changes in properties and geometry as trade-offs between molecular objectives, with less discussion of the latter effect.

Redocked Vina evaluates a molecule after an external pose search. A molecule can obtain a favorable score after redocking even when the generator initially places it poorly. QED and synthetic accessibility do not evaluate this placement, and the added geometry reward mainly checks intramolecular structure. Section 4.4 acknowledges the generated-pose issue. Improving molecular proposals for a docking program and improving directly generated three-dimensional complexes are distinct contributions. The current method more directly supports the former. The paper uses the same reward to update coordinates and discrete features jointly, without fully explaining the independent role of coordinate updates in this objective.

Questions for authors
  1. Under the decomposition in Equation 8, changing reward weights affects both objective preferences and the strength of directional updates relative to drift regularization. How do the authors interpret these two effects? How should the paper’s main conclusions about property–geometry trade-offs be understood in relation to them?

  2. When the primary feedback comes from redocking, what specific objective is served by jointly updating continuous coordinates? Is the intended improvement in molecular proposals for subsequent docking, or in the ability to generate plausible poses directly within the pocket?

  3. The weighted reward in Equation 7 allows chemical validity to be traded against property rewards. Is it intentional that an invalid molecule can receive a higher total reward than a valid molecule with low property scores? What design preference is this ordering intended to express?

2 comments

We appreciate the reviewer’s careful assessment and constructive suggestions. We have responded to all points and carried out additional experiments, which we believe further strengthened our paper.

Score-only vs. redocked Vina Reward: We agree with the reviewer that feedback from redocking optimizes molecular proposals for subsequent docking while Vina score-only reward directly optimizes for plausible ligand poses. We thank the reviewer for highlighting this angle and trained all Flowr.NFT configurations presented in Figure 3 and Table 1 with Vina score-only instead of redocked Vina reward.

Indeed, across all four configurations, this improves score-only Vina and intermolecular PoseBusters validity on the SPINDR test set and for zero-shot transfer to CrossDocked2020. For FlowR.NFT_multi, score-only Vina improves from −7.43 to −8.73 kcal/mol on SPINDR and from −1.14 to −4.49 kcal/mol on CrossDocked2020. Intermolecular validity increases from 0.90 to 0.98 and from 0.58 to 0.80, respectively. Redocked Vina remains favourable, although weaker than with the original reward: −9.96 versus −10.14 kcal/mol on SPINDR and −8.63 versus −9.10 kcal/mol on CrossDocked2020. QED improves on both datasets, while SA decreases. The Vina+QED+Geom configuration achieves a stronger pose-quality trade-off on CrossDocked2020: score-only Vina reaches −7.15 kcal/mol, while intermolecular validity reaches 0.9 and redocked Vina reaches −9.10 kcal/mol, although SA is lower at 0.53. We will present the full set of results corresponding to the current Figure 3 and Table 1 with score-only Vina in the camera-ready version of this paper.

Coordinate NFT loss for geometry-independent reward: Redocked Vina is not entirely geometry-independent. Standard docking changes ligand position, orientation and torsion angles of rotatable bonds while retaining bond lengths and angles. Therefore, it remains sensitive to generated intramolecular geometry, although it does not directly reward the initial placement of the ligand inside the pocket.

For coordinate-independent rewards such as QED, our motivation to include the coordinate term in the NFT loss is to align the joint molecular distribution. Changes in molecular composition and connectivity can change the corresponding distribution of plausible geometries. FlowR’s SEMLA backbone couples invariant and equivariant features, so joint updates allow coordinate predictions to adapt alongside the changing molecular graphs. This motivates retaining the coordinate loss.

Objective composition vs. update strength: Allocating reward weight to an additional property introduces feedback for that property while reducing the weights of existing objectives, without changing the drift regularisation term. Our experiments characterize property trade-offs but do not separate the benefit of the additional feedback from reduced pressure on other objectives. The matched neutral-component control proposed in Appendix A would help distinguish these effects. We therefore interpret the observed trade-offs as outcomes of complete fine-tuning configurations, rather than attributing them exclusively to conflicting molecular objectives. We will clarify this in the manuscript.

Reward ordering of invalid and valid molecules: The reviewer’s observation that our additive reward permits an invalid molecule to receive a higher reward than a valid molecule with sufficiently poor property scores is correct. It treats validity as a soft objective rather than enforcing a strict validity-first ordering. This design was motivated by a preliminary QED experiment, where we tested a validity-gated ri=min(rv,i,rp,i)r_i= min(r_{v,i}, r_{p,i}), where rv,i∈0,1r_{v,i}\in{0,1} indicates chemical validity and rp,i∈[0,1]r_{p,i}\in[0,1] is the normalized property reward. This assigns zero reward to every invalid molecule and resulted in validity collapse. We will clarify the permitted reward ordering and empirically motivate the additive reward formulation.

Reviewer_ik3BOpenReview ↗

Acknowledgment of Improvements; Novelty Concerns Remain

We thank the authors for their detailed rebuttal and the substantial additional experiments, particularly the score-only Vina results.

Score-only Vina reward. The new experiments confirm that replacing redocked Vina with score-only Vina improves both score-only Vina energy and intermolecular PoseBusters validity across all configurations and both evaluation datasets. This directly addresses our concern that redocked Vina optimizes molecular proposals for a docking program rather than the quality of directly generated poses. We consider this a meaningful improvement and encourage the authors to adopt this formulation in the final manuscript.

Coordinate updates for geometry-independent rewards. We appreciate the clarification that FlowR’s SEMLA backbone couples invariant and equivariant features, providing a reasonable motivation for retaining the coordinate loss term even under graph-only rewards such as QED. The argument that changes in molecular composition alter the distribution of plausible geometries is sensible, though an ablation removing the coordinate term for graph-only rewards would have made this case more convincing.

Objective composition vs. update strength. We note that the authors now agree that their experiments do not separate the benefit of additional property feedback from the reduced pressure on existing objectives caused by weight reallocation. The matched neutral-component control described in Appendix A (Eq. 44) was designed precisely for this purpose. Without running this control, the multi-objective analysis in Section 4.3 remains a characterization of complete fine-tuning configurations rather than an identification of which reward components are responsible for the observed trade-offs. Clarifying the language in the manuscript is welcome but does not substitute for the ablation itself.

Reward ordering of invalid and valid molecules. The additional validity-gated experiment showing validity collapse provides useful empirical motivation for the additive reward design. This addresses our question adequately.

Prior-art overlap with MolFORM v3. We have reviewed the public comment by the MolFORM authors and the authors’ response. The authors acknowledge that the continuous-coordinate branch construction (Eq. 1), the categorical logit-space branch construction (Eq. 3), and the joint reward-weighted objective (Eq. 5) overlap with the formulation in MolFORM v3 (arXiv:2507.05503v3, publicly available February 2026). These equations constitute the core of Section 3.1, and the first listed contribution of the paper is “develop FlowR.NFT by adapting DiffusionNFT to the joint continuous and categorical representations in molecular generation.” The authors’ commitment to acknowledge this overlap is appropriate, but it substantially reduces the methodological novelty of this contribution, since the joint continuous-categorical NFT adaptation was already described in publicly available prior work.

Overall assessment. After accounting for the prior-art overlap, the paper’s primary remaining contributions are: (1) the validity-aware reward formulation with neutral completion and the analysis of validity collapse, and (2) the systematic experimental study of reward compositions across SPINDR and CrossDocked2020. Both are valuable. However, the validity-aware formulation, while well-motivated by the gradient decomposition in Eq. 8, is a relatively focused design choice within an existing framework. The multi-objective study, as the authors now acknowledge, characterizes outcomes of complete configurations without isolating the mechanisms behind the observed trade-offs.

We appreciate the effort the authors have put into their response and the additional experiments. The score-only Vina results and the validity-gated ablation strengthen the paper. Nevertheless, with the reduced novelty of the NFT adaptation and the absence of the neutral-component control ablation, we believe the overall contribution does not yet meet the threshold for acceptance. We maintain our rating of Weak Reject.

Reviewer_NpGGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents an online reward‑aware fine‑tuning framework, i.e., FlowR.NFT. It is built upon DiffusionNFT for pocket‑conditioned 3D ligand flow‑matching generators. Standard online reward fine‑tuning for molecular generation often suffers from severe chemical‑validity collapse, i.e., if chemically invalid molecules are simply excluded from loss computation, the model shifts probability mass toward invalid structures to inflate average reward over valid samples. To resolve this issue, the authors develop a validity‑aware composite reward formulation. It provides both mathematical analysis and empirical evidence showing why simply discarding invalid molecules leads to chemical‑validity collapse, and proposes a neutral‑completion validity‑aware reward strategy. Multi‑objective reward‑weight sweeps reveal clear trade‑offs between affinity, drug‑likeness, synthetic accessibility and geometry. While intramolecular chemical quality is well‑maintained, intermolecular pose plausibility degrades after alignment, which remains an open challenge. Further work on pose‑aware reward components and adaptive multi‑objective weighting would strengthen real‑world applicability for de‑novo drug design.

Critical strengths

The paper clearly diagnoses the validity‑collapse risk when excluding invalid molecules in online reward fine‑tuning for 3D molecular generative models, a widely observed but under‑analysed failure mode in structure‑based drug design.

It provides gradient‑space mathematical decomposition explaining the difference between sample exclusion versus neutral‑reward completion. It rigorously shows why setting undefined‑property rewards to 1/2 preserves drift regularization without injecting misleading alignment gradients.

Critical weaknesses
  1. The reward set does not effectively preserve intermolecular ligand‑pocket pose quality. The redocked Vina improves but raw generated pose metrics degrade substantially. No modified reward design is proposed to mitigate this key limitation.

  2. All fine‑tuning uses SPINDR only. Although zero‑shot transfer to CrossDocked2020 is tested, there is no study of behaviour when fine‑tuning on mixed or more heterogeneous pocket datasets.

  3. All assessments are computational in‑silico metrics, like docking, QED, SA, PoseBusters. No wet‑lab synthesis or binding assay results to verify real‑world performance of optimized generated ligands.

Questions for authors
  1. Intermolecular pose validity worsens significantly after fine‑tuning, even with good intramolecular geometry. The authors are suggested to directly add intermolecular PoseBusters criteria into the composite reward.

  2. All fine‑tuning is performed on the SPINDR dataset. How would model behaviour change if fine‑tuning is performed on more heterogeneous pockets with broader protein families? Are there risks of reward‑overfitting to SPINDR pocket distributions?

  3. In your setup, Vina docking score serves as a proxy for true binding affinity. Vina has well‑known approximations and biases. How would FlowR.NFT behave if fed noisy or systematically biased proxy reward signals during online fine‑tuning?

  4. Reward weights are manually swept and selected. Could you introduce a Pareto‑front optimization procedure to automatically identify a set of non‑dominated reward‑weight configurations without extensive manual grid searching?

2 comments

We thank the reviewer for their detailed feedback and suggestions. We address all comments below report additional experiments conducted in response to the suggestions, which we believe have further strengthen the paper.

Intermolecular pose validity: We thank the reviewer for the suggestion to modify the reward to mitigate degraded intermolecular pose validity (and Vina score-only results) on CrossDocked2020 zero-short transfer. When evaluating on the SPINDR test set, the intermolecular validity is at least 0.90 for all our post-training configurations and also Vina score-only improves, see Tab. 3. Since we used docked Vina in the optimization objective, the pose was not directly rewarded in the presented experiments.

A direct way to integrate pose reward is to use raw Vina score-only as reward. This improves PB inter-molecular reward and raw Vina score-only on the SPINDR test set and zero-shot CrossDocked2020 transfer. For FlowR.NFT_multi, intermolecular validity increases from 0.90 to 0.98 on SPINDR and from 0.58 to 0.80 on CrossDocked2020. Overall PB validity improves from 0.87 to 0.93 and from 0.56 to 0.73, respectively. Score-only Vina improves from −7.43 to −8.73 kcal/mol on SPINDR and from −1.14 to −4.49 kcal/mol on CrossDocked2020. These improvements involve trade-offs: redocked Vina becomes slightly less favourable on SPINDR (−10.14 to −9.96) and CrossDocked2020 (−9.10 to −8.63), and SA decreases, while QED improves on both datasets. The Vina+QED+Geom configuration achieves a stronger pose-quality trade-off on CrossDocked2020: score-only Vina reaches −7.15 kcal/mol, while intermolecular validity reaches 0.9 and redocked Vina reaches −9.10 kcal/mol, although SA is lower at 0.53. We will present the full set of results for all of the configurations presented in Figure 3 and Table 1 in the camera-ready version of this paper.

Manually selected rewards: While the objective trade-offs that we have established through our sweeps can guide the choice of reward weights, we agree that automating the weight selection would improve practicality. For practical optimisation, we favour maximising affinity subject to specified molecular-quality constraints, with weights adapted according to constraint violations.

We conducted first experiments adapting Constrained Flow Optimization (Gutjahr et al., 2026) to dynamically adjust the reward weights in Eq. 7. We optimise score-only Vina subject to mean RDKit validity ≥0.95, QED ≥0.50, SA ≥0.59, and PB geometry score ≥0.98. The resulting model meets these thresholds on both SPINDR and CrossDocked2020, achieving score-only/redocked Vina scores of −9.73/−10.22 kcal/mol on SPINDR and −6.51/−8.77 kcal/mol on CrossDocked2020 (zero-shot transfer). These initial results support adaptive weighting as a practical extension of our reward-design study. We will include the method and experimental results in the appendix of the revised manuscript.

Fine-tuning on SPINDR: We thank the reviewer for highlighting the potential dependence on the fine-tuning pocket distribution. We selected SPINDR because it provides curated, structurally refined experimental protein–ligand complexes and retains PLINDER’s splits designed to reduce train–test leakage.

CrossDocked2020 serves a complementary benchmarking role. Its construction expands ligand–receptor combinations by docking ligands into related binding sites. Its larger number of examples therefore does not, by itself, establish broader protein-family or pocket coverage. We have evaluated our models on CrossDocked for comparison with established methods and recognize that evaluation without additional fine-tuning addresses a different question from varying the fine-tuning distribution.

We agree that studying such broader distributions would help assess sensitivity to pocket composition. Our current experiments establish results for SPINDR-based fine-tuning, and we will clarify in the manuscript that robustness to broader protein pocket distributions remains an open question.

Robustness to imperfect proxy rewards: FlowR.NFT follows the supplied reward. Random noise can weaken the alignment signal, while systematic, molecule-dependent biases can favour proxy exploitation over genuine affinity improvement. Within-pocket normalization removes systematic offsets and rescales rewards, but does not correct ranking errors. We have not tested controlled reward perturbations beyond the random shuffling of rewards when training on valid molecules only which mitigates validity collapse and, as expected, does not lead to reward improvement (Figure 2). The results in the paper establish gains in Vina scores, but robustness testing and independent affinity evaluation remain future work.

No wet-lab assessment: While experimental validation would be valuable, our study focuses on reward design for online post-training and evaluates its effects using computational benchmarks, wet-lab validation is beyond its scope.

Sven Gutjahr, Riccardo De Santi, Luca Schaufelberger, Kjell Jorner, Andreas Krause: Constrained Flow Optimization via Sequential Fine-Tuning for Molecular Design; International Conference on Machine Learning, 2026

Comments on the paper

1 comment
~Daiheng_Zhang1OpenReview ↗

Prior-art attribution concern: MolFORM v3 (arXiv:2507.05503v3)

I am an author of MolFORM and would like to raise an attribution/novelty concern about the MolFORM-RL formulation (Sec. 3.5, Eq. (5)) in arXiv:2507.05503v3 (Feb 25, 2026; https://arxiv.org/abs/2507.05503v3), which was publicly available before the ICLR 2027 deadline. This refers to v3, not the current arXiv version, and is limited to the online negative-aware fine-tuning mechanism that jointly optimizes continuous coordinates and categorical atom types, not the base MolFORM generator or its offline DPO formulation. I recognize that the general negative-aware fine-tuning principle originates from DiffusionNFT and do not claim ownership of it.

Specific overlaps:

  1. Continuous branches. MolFORM v3 builds positive/negative clean-coordinate predictions as x_old ± β(x_θ − x_old). FlowR.NFT Sec. 3.1, Eq. (1), appears to use the same construction with a coefficient β_c.
  2. Categorical logit-space branches. MolFORM v3 applies the same construction to atom-type logits, ℓ_old ± β(ℓ_θ − ℓ_old), with cross-entropy on these logits. FlowR.NFT Eq. (3) appears to use the corresponding construction, softmax[a_old,c ± β_d(a_θ,c − a_old,c)]; its Appendix A (Eqs. (23)–(24)) writes it in the same interpolation form as v3, and its stated reason for working in logit space (extrapolating probabilities can leave the simplex) also appears similar to v3's. Although FlowR.NFT includes additional categorical channels, the atom-type logit-space construction was already described in v3.
  3. Joint reward-weighted objective. MolFORM v3 Eq. (5) combines continuous-position and categorical cross-entropy losses across the positive/negative branches, weighted by r and (1 − r). FlowR.NFT Eqs. (2)–(5) appear to share this structure, differing in modality-specific coefficients and the categorical channels included.

The Introduction (p. 2) states that FlowR.NFT is developed "by adapting DiffusionNFT to the joint continuous and categorical representations in molecular generation," and Sec. 3.1 (pp. 3–4) gives Eqs. (1)–(5). Related Work (p. 3) cites MolFORM as a flow-matching generator and an offline preference-alignment application, but does not discuss the online continuous–categorical NFT formulation in v3. This does not establish which version the authors read. I would appreciate clarification on how the submission distinguishes its formulation from the one already public in v3.

MolFORM v3 also studied online Vina/QED/SA reward combinations. FlowR.NFT's analysis of validity collapse, invalid-sample handling, geometric plausibility, and reward trade-offs may be independent contributions.

Checkpoints and sampling/evaluation artifacts: https://huggingface.co/daiheng/MolFORM. The source code was public at https://github.com/daiheng-zhang/SBDD-MolFORM (as noted in v3) but has since been made private.

Disclosure: I am also an author of a related ICLR 2027 submission; this comment is based solely on the publicly available MolFORM v3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.