- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
A framework for training compact generators on discrete outputs. Samples are scored using various constraints and the resulting is optimized with antithetic evolution strategies. The generator allows for an entire graph to be generated in a single evaluation. The method is tested on standard datasets QM9 and ZINC250K.
Critical strengths
The one-step inference efficiency is shown through the VU yields, and also demonstrates overall training efficiency.
Critical weaknesses
Related works section not included in the main text, which would be useful for more context regarding the methodology and contributions. VU yield performance is mediocre. The strength in training budget and comparison does not compare to the stronger performers for VU yield. In particular, other one-step methods like MoFlow and the autoregressive methods are not mentioned. Further testing in various molecular objectives would be useful for more context regarding the quality of the generated molecules. Figure of methodology is not helpful and provides minimal information regarding the contributions, and overall presentation of the information is extremely dense and difficult to parse through.
Questions for authors
What exactly is fitness shaping for the experiments? What/Why do the molecular experiments use centered ranks? Additional information regarding the why and how to select the fitness would be great. Is there a specific reason related works was pushed to the appendix? It provides a lot of context and hurts the clarity of the methodology.
2 comments
Thank you for your response. I have no further questions.
Thank you for highlighting the presentation issues and for the constructive feedback. We agree that the context and supporting evidence needed to be easier to find. We have revised the manuscript (see the revised PDF) accordingly and welcome any further suggestions for improving it.
Our evaluation covers progressively larger molecular graphs: up to 9 heavy atoms on QM9, 38 on ZINC250K, and 88 on GuacaMol in our experiments. Larger graphs require coordinating more atom and bond decisions while satisfying chemical constraints. GuacaMol therefore extends the evaluation to larger molecular graphs. We also evaluate scaffold completion as a constrained molecular application (Appendix E). Beyond chemistry, we test neural architecture generation and examine both the benefits and limitations of the training framework.
1. Related work
We have added a Related Work section immediately after the introduction, while retaining the more detailed background and related work in Appendix B.
2. VU yield performance
Figure 2 and Appendix D compare EGO with autoregressive, iterative, and one-step generators. EGO achieves VU yields of 78.21% on QM9, 90.66% on ZINC250K, and 77.61% on GuacaMol. Its advantage concerns the balance between yield and inference cost. Compared with one-step methods such as CFM-ECLD, EGO achieves approximately 53× and 208× higher VU per estimated dense inference operation on QM9 and ZINC250K, respectively, despite lower absolute VU.
Specifically regarding MoFlow, we thank the reviewer for prompting a closer examination of its evaluation protocol. Its published results include validity correction after graph generation, an aspect that we did not make sufficiently clear in the text. We evaluated public MoFlow checkpoints without this post-hoc validity correction, using five sampling runs of 10,000 proposals per dataset. On QM9, MoFlow achieves 90.19% VU, compared with 78.21% for EGO. On ZINC250K, MoFlow achieves 81.59% VU, compared with 90.66% for EGO. The same pattern holds for validity: 96.04% for MoFlow versus 93.46% for EGO on QM9, and 81.87% versus 94.02% on ZINC250K.
Furthermore, the evaluated MoFlow models contain 2.65 million and 130.35 million parameters on QM9 and ZINC250K, respectively, compared with 1.46 million and 6.32 million for EGO. MoFlow therefore has approximately 1.82× and 20.62× as many parameters.
Accordingly, we have updated the comparison tables (Appendix D.2 and D.3), Figure 2, and the main text to distinguish the correction protocols and make these quality and cost trade-offs more explicit. We welcome further suggestions to improve the clarity and completeness of the comparison.
3. Molecular quality beyond validity
The scaffold completion experiments in Section 5.2 and Appendix E jointly require QED ≥ 0.7, SA ≤ 4, and no additional PAINS alert types relative to the scaffold.
QED, the Quantitative Estimate of Drug-likeness, summarizes physicochemical properties associated with typical oral drugs. SA, Synthetic Accessibility, estimates synthesis difficulty. PAINS (Pan-Assay Interference Compounds) filters flag structural motifs associated with potential assay interference.
At 100,000 proposals, EGO produces 20,245 distinct qualifying completions, compared with 16,684 for MoLeR. These three screening criteria are neither explicitly rewarded during training nor used for checkpoint selection. We consider the ability to satisfy them jointly additional evidence of generation quality beyond the explicit training criteria. These scores could also be incorporated directly into the fitness in future applications.
4. Methodology figure and overall presentation
We have also redrawn Figure 1 from scratch to make the method easier to follow. It now separates training from inference and illustrates the four stages: perturb, generate, score, and update. A revised caption and a short explanation in the main text guide the reader through the method.
5. Fitness shaping
Fitness shaping converts scores into ES update utilities. For the main molecular runs, centered ranks preserve candidate ordering while reducing sensitivity to score magnitudes and outliers. Rarity is ranked separately to control its influence.
A key advantage of our method is its support for non-differentiable criteria on hard outputs. This avoids designing a differentiable surrogate for each chemical constraint or diversity reward. We select components according to the task: distribution matching for reference similarity, chemical penalties for feasibility, and optional rarity for exploration. Appendix C.1 specifies the fixed weights and ranking procedure.
Scaffold completion ranks several components separately (Appendix C.2), whereas NAS uses mean-centered raw fitness without rank shaping (Appendix C.3).