acceptodds
Under review as a conference paper at ICLR 2027

Elite-Weighted Supervised Fine-tuning for Goal-Directed Molecular Optimization

Abstract

Goal-directed optimization is essential for steering molecular generators to propose candidates with desired properties. However, it is often implemented with policy-gradient reinforcement learning, which requires a generation-trajectory log-probability whose form depends on the model architecture and generation procedure. This makes an optimizer difficult to reuse across architectures and conditional generative designs. Supervised fine-tuning needs none of that machinery, but its update is driven by a fixed dataset, so the reward never enters the update. We introduce Elite-Weighted Supervised Fine-tuning (EW-SFT), which uses reward to guide elite selection of high-scoring molecules, and updates the model by its own pretraining loss on that set. Because the update consumes only scored molecules and the model's native loss, EW-SFT is a general-purpose formulation for goal-directed molecular optimization. We evaluate EW-SFT applied to three leading molecular generators (SAFE-GPT, an autoregressive model; GenMol, a masked diffusion model; and InVirtuoGen, a discrete flow model) across de novo, motif-extension, and linker-design tasks. Under a fixed budget of 3D shape alignment oracle calls on two kinase reference compounds, we demonstrate EW-SFT consistently outperforms the corresponding native optimizers. Furthermore, it improves goal-directed optimization under a 2D similarity oracle on four held-out references and achieves comparable performance on a sample-efficiency benchmark without a trajectory-level RL formulation. Ablations show that reward information is passed primarily through elite selection, rather than through continuous weighting within the selected set. These results demonstrate that EW-SFT is a unified and effective optimizer across molecular generators, design constraints, references, and oracles.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Molecular generation, goal-directed optimization, supervised fine-tuning, reinforcement learning, drug discovery

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jUtgOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces EW-SFT, which performs goal-directed molecular optimization through reward-guided elite selection and native-loss fine-tuning. The method is applied to SAFE-GPT, GenMol, and InVirtuoGen, and evaluated on de novo, motif-extension, and linker-design tasks.

Critical strengths

The proposed method is conceptually simple and avoids some complex machinery. The implementation and experimental settings are described in detail. The experimental scope is relatively broad.

Critical weaknesses
  1. The methodological novelty may be limited. It seems the method uses “elite+binary”. Under this setting, the method may just select top k molecules based on the reward and then perform standard fine-tuning. In other words, the reward may only determine whether the molecule is selected, and is not involved in weighting the gradient update. This design appears to be the core mechanism of previous hill-climbing methods. Also, in terms of implementation details, the proposed method appears to be similar to GEGL, which uses genetic search, a max-reward queue, and unweighted imitation as well. Then, the key difference of this work from previous studies would be using each architecture’s native pretraining loss and extending the approach to different architectures. It is unclear if these could be considered as sufficient methodological novelty. Additionally, the native pretraining loss may have different meanings in different architectures. For example, in GenMol and InVirtuoGen, the native pretraining loss may not be the exact negative log-likelihood of generated molecules. Then, optimizing or reducing these native pretraining losses may not necessarily imply a higher probability of generating high-reward molecules as intended. Even though the native pretraining loss could be reused, applying the full pipeline to different architectures may not be easy, which may still need lots of architecture-specific designs.

  2. The experimental evidence may not fully support the advantage or effectiveness of the proposed method. The ablation results may suggest that native-loss fine-tuning is not as effective as expected. When isolated from genetic search, native-loss fine-tuning seems to fail to improve the generator and even decrease the performance in GenMol and InVirtuoGen. It is likely that the major improvement of the method is from the external genetic search rather than the proposed native-loss update. Also, genetic search may already produce better training samples based on the reward and the contribution of elite selection may be unclear. There is no ablation of “G+batch+binary” to support the usefulness of elite selection. What’s more, the reported advantage of the proposed method may be due to the additional oracle calls. Under matched oracle-call budgets, the performance is close to or even worse than baselines.

Questions for authors
  1. What are the substantial methodological contributions of this study?

  2. Could the authors provide more convincing evidence to support the effectiveness, necessity, or advantage of the proposed method?

2 comments
AuthorsOpenReview ↗

Response to Reviewer jUtg

The additional experiments and controls warrant reassessing soundness.

1. Methodological contribution and native-loss interpretation

Our method is designed to investigate a shared reward-guided native-loss update across architectures, learning objectives, and generative designs. To avoid deriving policies for each generation procedure, we learn from completed, scored molecules, separating proposal source, reward-based selection, and within-set emphasis. Ablations identify search + elite selection + binary learning as the preferred shared configuration. Its simplicity is an outcome of this investigation.

GEGL's unweighted likelihood refitting is a working special case at the learning-principle level. Structural similarity of the final best variant does not make our original formulation or findings a transplantation of GEGL. Besides, our generators perform conditional completion/regeneration as genetic operations, while global elite selection differs from its separate apprentice/expert queues. Within GEGL itself, using EW-SFT significantly improves mean AUC Top-10 from 0.712 to 0.736 (p=0.0025) over native dual-queue sampling at 10k calls. The design difference matters under a limited budget (see implementation details and table in Reviewer Exm6, Response 1: seven-task controlled comparison).

Meaning of the native loss: Sections 2 and 3.2 distinguish NELBO from exact NLL. Reducing NELBO lowers an upper bound on NLL but does not guarantee increased exact likelihood or expected reward. Our claim is empirical: native-loss updating on reward-guided genetic samples improves optimization outcomes.

2. Updating contributes beyond genetic search

SFT-only removes genetic proposals and therefore changes the training-data source. Its weak performance shows the importance of those proposals. The GA-only comparison isolates the contribution of model updating while retaining the genetic-operator design.

Paired comparisons across 2 references, 3 tasks, and 3 seeds give:

BackboneMean Top-1k gain: EW-SFT − GA-onlyHolm-adjusted p
SAFE-GPT+0.0200.014
GenMol+0.0120.041
InVirtuoGen+0.0380.012

All three backbones show significant mean gains. These controls directly establish gains from learning beyond genetic search alone.

3. Requested G+batch+binary control

Holding the genetic-operator design and binary weighting fixed, the added control gives:

BackboneMean Top-1k difference: elite − batchHolm-adjusted p
SAFE-GPT+0.0110.037
GenMol+0.0100.057

Elite selection improves the mean on both backbones, significantly for SAFE-GPT. This directly tests selection beyond training on genetically generated batches. Together with the weighting ablations, it supports elite+binary as an effective configuration without continuous reward weighting, as we already mentioned in the manuscript "the reward signal needs to enter somewhere".

4. Oracle calls and effectiveness

We disagree with the suggestion that EW-SFT's improvement is due to additional oracle calls. Both methods receive the same 20k-call budget and the same novelty-based stopping rule. The native optimizers trigger this criterion because their search fails to maintain sufficient novelty, whereas EW-SFT continues generating previously unseen molecules. This is a difference in optimization behavior, not an unfair allocation of oracle calls.

Table 10 additionally reports the matched-call comparison, although truncating EW-SFT at another optimizer's stopping point imposes a termination condition that EW-SFT itself did not trigger. Even under this comparison, EW-SFT achieves higher Top-1k means in 13 of 16 cells and positive average gains for all three backbones.

EW-SFT continues discovering high-scoring molecules after native stopping, truncation excludes this demonstrated benefit (Reviewer Exm6, Response 2: counts before/after cutoff).

The primary evaluation asks which method finds better molecules under the same maximum budget and stopping rules. The ability to continue discovering high-scoring molecules is part of that evaluation, not a confounding factor that should be removed. PMO complements this evidence with competitive performance across broader objectives using a simpler update (see Reviewer Exm6, Response 2).

In 3D similarity tasks, among runs meeting each criterion, EW-SFT reaches GA-only's final Top-1k mean or exceeds its best-ever score using median 62% or 32% of paired GA-only actual calls (Reviewer YEWo, Response 3).

On seven non-similarity PMO oracles (three seeds, exactly 10k calls, no early stopping), mean AUC Top-10 rises from 0.463 to 0.732 for SAFE-GPT and 0.902 to 0.911 for GenMol (Reviewer Exm6, Response 3). All seven SAFE-GPT means improve, with less runtime (Reviewer YEWo, Response 3).

We ask the reviewer to reassess soundness and effectiveness using these supplementary results.

I appreciate the authors' efforts in the rebuttal. It somewhat improves the soundness. However, overall, the key concerns in my initial review seem not to have been fully addressed.

  1. The core method or algorithm seems to be very similar to existing ones.

  2. Stronger theoretical justification is needed for the use of the native loss, rather than mainly relying on empirical results. This concerns the theoretical correctness.

  3. On GenMol, the p-value is 0.057, which seems not to be statistically significant. Does this mean elite selection may not be necessary for some backbones?

  4. Its generalizability to other architectures may still not be clear. But this is not the current major concern.

Given this, I will keep my original assessment for now.

Reviewer_Exm6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes Elite-Weighted Supervised Fine-Tuning (EW-SFT) for goal-directed molecular optimization. It combines genetic search, a rolling elite buffer, and supervised updates using each generator’s native training loss. Experiments cover autoregressive, masked diffusion, and discrete flow generators across de novo and fragment-constrained tasks. The method improves similarity optimization and achieves competitive PMO performance with a simpler update procedure.

Critical strengths
  1. The update reuses each generator’s native training loss and avoids trajectory-level policy-gradient machinery, offering a straightforward implementation strategy.
  2. The authors include GA-only and SFT-only baselines and report matched-oracle-call comparisons, allowing readers to assess the role of genetic search and continued optimization.
Critical weaknesses
  1. As the authors acknowledge, GEGL already combines neural proposals, genetic improvement, persistent high-reward queues, and unweighted maximum-likelihood training on retained molecules. EW-SFT extends this framework through native-loss updates, model-specific genetic operators, and constrained generation interfaces, while separately examining selection and within-set weighting. These extensions are useful, but the final elite+binary rule retains the same search–selection–supervised-learning structure. The paper does not sufficiently establish which changes constitute a substantive algorithmic advance beyond adapting elite refitting to newer generators. A controlled GEGL-style comparison that isolates these changes would strengthen the novelty claim.

  2. Cross-architecture evaluation primarily concerns two 3D reference compounds; the additional 2D and PMO experiments evaluate EW-SFT only with InVirtuoGen. In Table 10, matching actual oracle calls reduces InVirtuoGen’s average gain from 0.079 to 0.011, although average gains remain positive for all three backbones. Both methods share the same early-stopping rule, so sustaining productive search is a legitimate advantage, but these results provide weaker evidence for uniformly improved performance at equal query counts. On cold-start PMO, EW-SFT scores 16.385 versus 16.674 for Genetic-PPO, with no statistically significant difference reported. This supports the authors’ claim of a competitive, simpler alternative, while leaving broader sample-efficiency advantages unestablished.

Questions for authors
  1. What methodological changes beyond substituting native losses and adapting generation interfaces distinguish EW-SFT from GEGL-style elite refitting?
  2. Do the benefits transfer to representative non-similarity objectives on SAFE-GPT and GenMol, particularly under matched oracle calls?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Exm6

1. Methodological contribution

Our method is designed to investigate how one shared method can fine-tune generators across model architectures, learning objectives, and generative designs. To reuse native losses without deriving a policy for each generation procedure, we learn from completed, scored molecules. The formulation is a normalized weighted sum of native losses: a reward-based selection gate times binary or continuous within-set emphasis. Proposal source is a separate choice. This makes reward-guided learning channels explicit and testable, rather than assuming elite refitting. Ablations identify genetic search + elite selection + binary native-loss learning as the empirically preferred shared configuration. Its simplicity is an outcome of this investigation.

GEGL studies genetic-expert exploration and neural-apprentice imitation, using an LSTM, external graph-genetic proposals, and likelihood refitting from two queues. Its unweighted exact-likelihood update is a working special case of our formulation at the learning-principle level. Methodologically, our generators perform conditional completion/regeneration as genetic operations, and our global elite selection differs from their separate genetic and model queues. Structural similarity of the final best variant does not make our original formulation or findings a transplantation of GEGL.

We tested EW-SFT on GEGL. We retained GEGL's LSTM, graph-genetic expert, 10k-call budget, and eight gradient steps per round, changing only the training-set selection. In this PMO implementation, native GEGL randomly samples 64 molecules from two Top-128 queues' union, EW-SFT uses the global Top-64 for learning. Across seven non-similarity oracles, using EW-SFT improves mean AUC Top-10 from 0.712 to 0.736 (paired p=0.0025, 3 seeds). Our global elite selection significantly improves over dual-queue sampling under the limited budget, even with a similar search-selection-learning structure.

OracleGEGL nativeGEGL + EW-SFT
gsk3b0.7730.808
jnk30.5830.678
drd20.9430.940
qed0.9400.941
isomers_c7h8n2o20.9250.932
isomers_c9h10n2o2pf2cl0.8200.851
valsartan_smarts0.0000.000
Mean0.7120.736

Our genetic operators also preserve anchors for hard motif/linker design, beyond GEGL's de novo and similarity-constrained settings. To our knowledge, this is the first goal-directed optimization study of generalist generators on this fragment-constrained benchmark, demonstrating substantial improvement headroom.

2. Matched calls, sustained search, efficiency, and PMO

Both methods receive the same 20k-call budget and stopping rule. Whether an optimizer continues generating novel candidates is determined by its own search behavior. Table 10 artificially truncates EW-SFT at the native stopping point, but EW-SFT still improves 13/16 means, with average gains of +0.039 SAFE-GPT, +0.012 GenMol, and +0.011 InVirtuoGen. Additional calls cannot explain all gains.

For early-stopped native runs, we count distinct molecules reaching the native final Top-1k mean, splitting EW-SFT at the paired native call count (means over cells and seeds).

Model / cellsNativeEW-SFT before cutoffEW-SFT after cutoff
SAFE-GPT / 2 motif cells4221401801
InVirtuoGen / all 6 cells4656251547

Continued search adds 1,801 and 1,547 unseen high-scoring molecules per run and improves final Top-1k means. Truncation excludes this demonstrated benefit, full-budget performance reflects the ability to sustain optimization under the same experimental constraints. PMO complements this optimization behavior by demonstrating competitive performance across broader objectives with a simpler update.

Broader efficiency. In 3D similarity task (Reviewer YEWo, Response 3), among runs meeting each criterion, EW-SFT reaches GA-only's final Top-1k mean or exceeds its best-ever score using median 62% or 32% of paired GA-only actual calls.

3. New non-similarity results on SAFE-GPT and GenMol

On seven non-similarity PMO oracles, all three seeds complete 10k calls. EW-SFT improves all seven SAFE-GPT AUC Top-10 means and the overall mean on both backbones, demonstrating gains at equal actual calls. As shown above, EW-SFT also significantly outperforms the native GEGL on these tasks.

OracleSAFE-GPT PPOSAFE-GPT EW-SFTGenMol GAGenMol EW-SFT
gsk3b0.4820.8430.9760.945
jnk30.2690.7290.8450.884
drd20.6350.9480.9950.995
qed0.9350.9370.9410.941
isomers_c7h8n2o20.3670.7720.9290.972
isomers_c9h10n2o2pf2cl0.5530.7400.8180.841
valsartan_smarts0.0000.1530.8120.800
Mean0.4630.7320.9020.911

I thank the authors for their response. However, I still have reservations about the methodological contribution and will therefore leave my score unchanged.

Reviewer_YEWoOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

Summary

This paper proposes Elite-Weighted Supervised Fine-tuning (EW-SFT) for molecular optimization. The method combines genetic search with fine-tuning on high-scoring molecules using each generator’s native training objective. Experiments cover three generative architectures and both unconstrained and fragment-constrained design tasks.

Critical strengths

Strengths

  • The method is simple and applicable across different generative architectures without requiring a trajectory-level policy-gradient formulation.
  • The experiments demonstrate consistent improvements over the corresponding native optimizers on similarity-based tasks, while achieving competitive performance on PMO.
  • The ablations provide useful insight into the respective roles of genetic search, elite selection, and within-set weighting.
Critical weaknesses

Weaknesses

  • The methodological novelty appears limited. Learning from high-reward samples is well established in GEGL [1], and Genetic GFN [2] also combines genetic search with reward-prioritized replay. Although their training objectives differ, the conceptual contribution beyond these existing approaches could be clearer.

[1] Ahn et al., Guiding Deep Molecular Optimization with Genetic Exploration, NeurIPS 2020.

[2] Kim et al., Genetic-guided GFlowNets for Sample Efficient Molecular Optimization, NeurIPS 2024.

Questions for authors

Questions

  • The performance gains over GenMol’s native genetic algorithm appear marginal. Could this be because EW-SFT shares much of its optimization mechanism with the existing genetic search?
  • On PMO, EW-SFT achieves comparable performance to Genetic-PPO with a simpler training objective. Does this simplicity translate into practical benefits, such as lower computational cost or less hyperparameter tuning?
1 comment
AuthorsOpenReview ↗

Response to Reviewer YEWo

1. Methodological contribution

Our method is designed to investigate how one shared method can fine-tune generators across architectures, learning objectives, and generative designs. To retain native losses without deriving a policy for each generation procedure, we learn from completed, scored molecules. Normalized loss weights separate a reward-based selection gate from binary or continuous within-set emphasis; proposal source is a separate choice. Ablations identify genetic search + elite selection + binary learning as the preferred shared configuration. Its simplicity is an outcome of the ablation.

GEGL's unweighted LSTM likelihood refitting is a working special case at the learning-principle level. But structural similarity of the final best variant does not make our original formulation or findings a transplantation of GEGL. Methodologically, our generators perform conditional completion/regeneration as genetic operations, and global elite selection differs from GEGL's separate apprentice/expert queues. We tested EW-SFT on GEGL: retaining its LSTM, graph-genetic expert and update schedule, EW-SFT's global Top-64 selection improves mean AUC Top-10 from 0.712 to 0.736 (p=0.0025) over native dual-queue sampling on seven non-similarity PMO tasks at 10k calls (see implementation details and table in Reviewer Exm6, Response 1). The design difference matters under a limited budget, even with a similar search-selection-learning structure.

Genetic GFN uses graph-based GA exploration, rank-based replay sampling, and GFlowNet policy optimization using trajectory balance loss with KL regularization. Neither prior work establishes our central finding: a shared update retaining autoregressive NLL or diffusion/flow NELBO across unconditional and hard fragment-constrained generation without deriving generation-trajectory likelihoods.

Our genetic operators also preserve anchors for hard motif/linker design, beyond GEGL's de novo and similarity-constrained settings. To our knowledge, this is the first goal-directed optimization study of generalist generators on this fragment-constrained benchmark, demonstrating substantial improvement headroom.

2. Gains beyond GenMol's native GA

GA is GenMol's native optimizer, and we deliberately retain it when applying EW-SFT. In Table 1, all six 3D-setting means improve, the +0.012 average gain is modest but statistically significant (Holm-adjusted p=0.041).

When GA is not the native optimizer, the same GA-only control on SAFE-GPT and InVirtuoGen gives larger mean gains, +0.020 (p=0.014) and +0.038 (p=0.012), respectively (see the table in our response to Reviewer jUtg, Response 2). GA-only itself outperforms their native optimizers on average in Table 2, yet EW-SFT improves further. Although GA is a well-recognized strong baseline, these controls establish complementary contribution of EW-SFT: Genetic search supplies useful proposals, while learning from selected proposals enhances subsequent optimization.

3. Practical benefits: implementation, runtime, and search progress

The update reuses native training code without trajectory likelihood derivation, importance ratios, or RL-specific clipping and KL terms. We just reused the EW-SFT hyperparameters from the 3D experiments in the additional PMO runs. The implementation has far fewer update-specific choices (learning rate and elite-buffer size K), versus 14 RL-related configuration entries in Genetic-PPO, including replay, clipping, KL/entropy settings and advantage mode. This reduces configuration and implementation requirements.

On seven non-similarity PMO oracles (three seeds, 21 runs in total), SAFE-GPT EW-SFT substantially improves the Top10-AUC mean from 0.463 to 0.732, winning all seven tasks (per-oracle results in our response to Reviewer Exm6, Response 3), while reducing average runtime:

SAFE-GPT optimizerAggregate runtimeMinutes/run
Native PPO5.16 h14.7
EW-SFT4.12 h11.8

We additionally compared frozen GA-only against EW-SFT in the main 3D similarity task (2 references, 3 tasks, 3 seeds, 18 paired runs/model; 20k-call budget). We measured the first call at which EW-SFT reached GA-only's final Top-1k mean or exceeded its best-ever single-molecule score.

ModelCalls to reach GA's final Top-1kCalls to exceed GA's best-ever
SAFE-GPT64%20%
GenMol62%37%
InVirtuoGen59%36%
All62%32%

Among runs meeting each criterion, the median calls required are 62% of the paired GA-only run's actual oracle calls to reach its final Top-1k mean and 32% to exceed its best-ever single-molecule score. These results show EW-SFT's practical benefits in search progress and attainable scores.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.