acceptodds
Under review as a conference paper at ICLR 2027

Joint Foundation Model for Synthesizable Molecular Generation and Property Prediction

Abstract

Molecular foundation models deliver strong property prediction from limited labels but produce encoders that cannot propose new molecules, while latent-variable generators are trained for reconstruction rather than transfer and decode unreliably away from their data. Design needs a third ingredient neither supplies: assurance that what is proposed can actually be synthesized. The model presented here, that is a generative version of the foundation model CheMeleon, puts all three on one representation. It pairs the descriptor-supervised CheMeleon D-MPNN with an autoregressive transformer decoder, optimizing Mordred descriptor regression and SMILES reconstruction jointly at foundation scale, so that a single -dimensional embedding serves as property representation, generative latent and search domain. A rectified flow fit to cached embeddings replaces the intractable Gaussian prior and doubles as a projection operator that keeps latent search decodable. Synthesizability enters the same space rather than a post hoc filter, as a differentiable head distilled from a retrosynthesis planner and, in a second arm, as hard projection onto routed analogs. Invertibility costs RMSE on property prediction across 30 MoleculeACE assays. The generative capabilities, tested on the GuacaMol benchmark, are competitive to state of the art models for both unconditional and goal-oriented generation tasks. Driven by property-head gradients under Langevin dynamics, the model proposes novel, potent, synthesizable candidates on MoleculeACE assays, each with an explicit synthetic route.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
foundation model, generative model, synthesizability, property prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2s9COpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  1. Unified Prediction and Generation: Jointly training models for molecular descriptor prediction and structural reconstruction enables a single latent representation to serve both property prediction and molecular generation tasks.
  2. Flow-Projected Latent Space Search: Leveraging Rectified Flow to learn the distribution of molecular representations, while employing projection techniques to enhance the decodability of candidate latent vectors during the optimization process.
  3. Incorporating Synthesizability into Optimization: By applying penalties based on synthesizability predictions or utilizing projections onto synthesizable analogs with explicit synthetic routes, the framework generates candidate molecules that balance novelty, predicted activity, and synthetic feasibility.
Critical strengths
  1. Unified design for prediction and generation: By jointly training descriptor prediction and molecular reconstruction, a single latent representation is enabled to simultaneously support property prediction, molecular generation, and optimization, with only a marginal decline in property prediction performance.
  2. Effective mechanism to address latent space mismatch: Rectified Flow is employed to handle both prior sampling and search projection, mitigating the issue where high-dimensional latent vectors deviate from the decodable region.
Critical weaknesses
  1. Lack of ablation studies on gradient guidance: The search process is influenced by gradient guidance, stochastic noise from Langevin updates, and the noise-addition/denoising operations within the flow projection. However, the paper does not verify the independent contribution of gradient guidance to the generated results through controlled comparisons.
Questions for authors

It is recommended to include an ablation study comparing the use and non-use of gradient guidance, while keeping other settings and the search budget constant, to verify the independent contribution of gradient guidance to the generated results.

1 comment

We thank the Reviewer for the positive comments about our work and we agree that an ablation study on the gradient guidance could further strengthen our claim.
The ablation study was conducted across the 30 MoleculeACE assays at fixed budget (30 generations with population of 300 molecules). The Table 10 in Appendix F sums up the results. The search with guidance on achieves better performances in the following metrics: best predicted potency throughout the 30 assays; margin over the potency value corresponding to the 90th quantile of the training set for each assay; the percentage of the generated molecules that satisfy the potency constraint (higher than q90); the overall success rate (molecules satisfying all criteria for valid demonstrations); distinct scaffolds. While the percentage of molecules clearing the q90 threshold is similar, it can be inferred that guidance helps the search going towards higher potent molecules, as evidenced from the best.\

The analysis carried out for Reviewer y1YN helped also in clarifying the role of guidance. In the “Reliability of the predictor” paragraph of Section 3.5, guidance is shown to be a key asset in driving the search towards molecules similar to unseen test-set molecules known experimentally to have higher activity.

Reviewer_9jmpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work builds on top the the CheMeleon small molecule encoder to build a latent variable generative model with joint synthesis planning. The representation is kept aligned to downstream property prediction and enables synthetic analog projection via PrexSyn incorporation.

Critical strengths
  • Well motivated work. Leveraging strong encoders to turn into a generative model makes a lot of sense and repurposes advances in representation learning
  • Light weight architecture though no training of speed benchmarks provided
Critical weaknesses
  • Limited benchmarking and evaluation. Missing comparisons to methods like MolMIM [1] which also created a fixed latent space for molecular optimization.
  • PreSyn is more of an add on and synth head alternative does not perform as well.
  • This work does not outperform older much simpler baselines for conditional design and does not benchmark against more recent methods

[1] https://arxiv.org/abs/2208.09016

Questions for authors
  • Why is the training corpus so small for a foundation model? Prior generative work GenMol [1] uses 1B compared to the 4M molecules used here. How does this model compare to GenMol for the conditional or property optimization tasks
  • How does this work compare to MolMIM [2] on the blackbox optimization tasks (table 2 and 3)

[1] https://arxiv.org/pdf/2501.06158 [2] https://arxiv.org/abs/2208.09016

1 comment

We thank the Reviewer for the suggestions about further comparing our work to other literature models. We would like to emphasize the importance of adding synthesizability within the generation loop, which is a key asset for guaranteeing the synthesizability of the generated molecules: imposing synthesizability as a constraint allows not only to have generated candidates that are closer to real chemistry, but also to propose molecules that can be directly tested in lab. The inclusion of PrexSyn into the generation loop allows in fact to have generated candidates with a proposed synthetic route, and this is not true for any of the models that are compared to this work, and neither for MolMIM and GenMol. More details on the synth head were given as also asked by the reviewer y1YN. Despite the difficulties of having a reliable synth predictor, its inclusion drove the synthesizability of the generated molecules from 30.1% to 49.9% and the number of valid demos is very close to the arm using PrexSyn (29/30 vs 30/30). Nevertheless, all the proposed candidates for our MoleculeACE test are synthesizable according to retrosynthesis planners.
Answers to the questions follows:
\

  • Question 1. More details about the choice of a small pre-training corpus are given in Appendix D. The model needed to be tested in the same pre-training configurations of standard CheMeleon, and using the released ChemBL split. Then the addition of MOSES was driven by a synthesizability experiment. Also, Tables 4 and 5 in Appendix C and D, suggest that the predictive performances were not enhanced when increasing the corpus from 1M to 4M. We do not exclude that increasing further the pretraining corpus can drive higher the design capabilities of the model making it an outlook of this work.
    A comparison with GenMol for the property optimization task is added to the new Section 3.4 “Further comparisons with literature models” and more details are provided in Appendix E. At fixed budget of 10K our model is just below GenMol and f-RAG (trained on a 1B corpus). However, as also shown in Figure 4, our model is budget limited. Indeed, raising the budget to 100K the score goes higher than the GenMol’s. Even though the scores at 10k and at 100k are not directly comparable, this is to show that our model largely improves when increasing the budget.
    \
  • Question 2. In Section 3.4 our model is compared to Table 2 and 3 of MolMIM. On the QED task of Table 2 our model scores 95% of success (compared to 94.6% of MolMIM), using two orders of magnitude less budget. We did not compare on the logP task of MolMIM’s table 2, since the authors explicitly report that such task largely exploits oracles. Comparing to Table 3 instead, our model achieves superior performances to MolMIM (randomly initialized) in terms of diversity and success rate, while getting lower novelty. The lower novelty is to be expected given the very small novelty metric of our pretraining dataset (around 3%) and our flow projection mechanism that re-anchors the candidates on manifold. Indeed, tuning the projection strength we can get very high in novelty (around 90%), while decreasing the success rate, as foreseen. We therefore retain the hard-projection regime (strong projection) as optimal, because it largely increases the robustness and the reliability of the decoder and of the predictor.
Reviewer_y1YNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a molecular foundation model that jointly learns property prediction and molecular generation through a shared latent representation. The method combines descriptor-supervised pretraining with molecular reconstruction, uses a rectified flow for latent-space sampling and optimization, and incorporates synthesizability constraints through a learned surrogate or synthesis-route projection.

Critical strengths
  • The paper addresses a relevant problem in molecular design, combining property prediction, molecular generation, and synthesizability constraints in a unified framework.
  • The proposed framework is intuitive and well motivated, particularly the use of a learned prior to support optimization in the foundation model's latent space.
  • The experiments are thorough, including 30 bioactivity prediction tasks and both unconditional and goal-directed generation.
  • The results demonstrate good molecular reconstruction and competitive generative performance, with only a relatively small degradation in downstream property prediction.
Critical weaknesses
  • I am not convinced by the main molecular design evaluation. The authors use the same learned property predictor to guide molecular optimization and evaluate the potency of the generated molecules. How can we know whether the generated molecules actually have the desired properties, rather than exploiting errors in the predictor? This is particularly concerning given the novelty constraints and the very high predicted potencies reported in Figure 5. The authors acknowledge this limitation, but I still consider it substantial. Furthermore, the predictive comparison in Table 4 uses full fine-tuning, while the default generation experiments use a frozen backbone.

  • I find the main success metric in Table 2 rather weak. The authors report success if they generate just one molecule per assay with predicted potency above the training-set 90th percentile, sufficient novelty, and a synthesizability score or route. Given the relatively large search budgets, how difficult is satisfying these criteria? The paper does not report success rates over generated molecules, diversity among successful candidates, or comparisons against simple sampling and screening baselines. Novelty is also measured only against the assay training molecules. Also, the reported 30/30 success requires additional fine-tuning and a doubled search budget for the two initially unsuccessful assays.

  • The methodological novelty appears limited. The proposed framework combines the existing CheMeleon encoder with a conventional autoregressive decoder, joint training objectives, rectified flow, latent-space optimization, and existing synthesizability methods. The paper doesn't really offer a substantial methodological contribution. The authors also do not sufficiently discuss closely related work on joint molecular generation and property prediction, such as UniGEM (ICLR 2025) and Hyformer (TMLR 2026).

  • Several important ablations and baselines are missing. Most notably, the authors do not compare joint pretraining against simply training a decoder on top of the original frozen CheMeleon encoder. The property-guided generation experiments also lack comparisons against simpler optimization approaches using the same property predictor.

  • The synthesizability evaluation is not entirely convincing. The optimization approach uses a distilled synthesizability head, but the authors report only its prediction loss. The paper is missing an evaluation its accuracy or actual retrosynthesis success on generated molecules. The enforcement approach uses PrexSyn to replace generated molecules with synthesizable analogs, but the paper provides little analysis of how much these replacements change the original molecules or their properties. I also don't understand why the differentiability of the synthesizability head is emphasized, given that Equation 3 uses only property-head gradients, while synthesizability enters the candidate ranking.

Questions for authors
  • I would like the authors to address my concerns about the property evaluation and the main MoleculeACE results. Can they provide independent evaluations of the generated molecules' properties, or additional evidence that the predictor remains reliable on these molecules? I consider this particularly important given the reliance on the same predictor for optimization and evaluation.

  • I would also like the authors to clarify the novelty of the proposed framework relative to existing joint generative-predictive models and provide more convincing evidence for the benefit of joint pretraining. In particular, I think a controlled comparison against a decoder trained on frozen CheMeleon embeddings, together with simpler molecular optimization baselines, would be valuable.

2 comments

We thank the Reviewer for the thorough analysis of our work and the precious comments. Answers to the Reviewer’s questions and comments follows:
\

  • Question 1. We understand the concern of the Reviewer, and we agree with him that in our case predictor exploitation must be handled with care. We specifically selected potency higher than the 90th quantile (and not higher than the maximum) to try limiting predictor exploitation. In addition, the candidate molecules are guaranteed to be synthesizable, thus representing real chemistry. Of course, these are just ways of making the prediction more robust, but they do not guarantee the predicted potency. There is no obvious way of guaranteeing that the predictions are all safe and sound unless an experimental campaign is carried out on the new molecules generated by the model. However, what we can do is assessing two things: (i) establish that the guidance drives the search towards molecules similar to unseen test set molecules known experimentally to have higher activity. Also that the search did not wander in the latent space, far from the chemical region of interest (which can happen if the search exploits errors in the predictor); (ii) provide uncertainty estimates and verify if the potency score stays higher than q90 within all the uncertainty range. Additionally, the predictor we are using inherits from the standard CheMeleon, that is state-of-the-art for molecular property predictions, at least as tested on the MoleculeACE benchmark. We added to the paper a full paragraph in which we discuss points (i) and (ii), i.e. the reliability of our high potency predictions (Section 3.5, “Reliability of the predictor”). The paragraph shows that the gradient guidance is effective in capturing genuine structure-activity relationships and, thanks to a baseline, it shows that such thing cannot arise at random. We then provide a Mondrian bin-conditional conformal prediction, assessing the overprediction based on unseen experimental points: in that way, error estimation and correction rest entirely on experimental measurements. This analysis confirms a positive result in the majority of the cases investigated.
    \
  • Question 2. The novelty of the work is now stated in a clearer way at the end of the Introduction. Our contribution is not that joint prediction-and-generation training is new, and we should not have implied otherwise. It is that an already published and independently validated predictive foundation model can be made invertible in place. Most importantly, we include synthesizability into the generation loop. That amounts to having proposed candidates that all are synthesizable according to a retrosynthesis planner. Conversely, the most common way of optimizing synthesizability is by optimizing scores (such as SAscore) that however do not imply actual synthesizability. Additionally, we provide evidence for the benefit of joint pretraining. It is shown that training only the decoder (no Mordred loss) the predictive performances decrease (Section 3.1 and Appendix C), as well as unconditional generation capabilities ( Section 3.2). A decoder cannot be attached to the frozen CheMeleon model already published because that would have a deterministic latent space, that has poorer performances than a stochastically smoothed embedding (Table 1). We found a bug in the finetuning of the joint model pretrained on the PubChem corpus: the new metrics are reported in Table 5 and 6 (Appendix C and D) and show that the joint model has mostly the same predictive performances as standard CheMeleon. Simpler optimization baselines (plain GA and CMA-ES) are tested now keeping fixed the predictor, to show that our high-dimensional and complex latent space requires the full machinery developed (Section 3.5, “Valid demonstrations”).

    Critical weaknesses:
    \
  • A virtual screening baseline is reported now showing that the search does substantial work on top of the screening (Section 3.5, “Valid demonstrations”, and Appendix F). Additionally, the screening has extremely poor performances applying the conformal correction developed for Question 1 ( “Reliability of predictor”, Section 3.5).
    \
  • More details about the synth head are provided (Appendix B, “Additional details on synthesizability”), showing that it is a good classifier but not a good regressor. Nevertheless, using it for scoring the population increases a lot the percentage of solvable molecules generated by the model. Tanimoto similarity of the PrexSyn routed analogs with the generated molecules is reported now, showing that only small tweakings are actually made (Section 3.5, “Valid demonstrations”, and Appendix B, “Additional details on synthesizability”). We agree that the differentiability of the synth head should not be emphasized, hence the term was removed from the text.

Thank you for answering my questions. I maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.