- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PhAME, a latent-diffusion framework for molecular editing conditioned on molecular properties or biological phenotypes. PhAME operates in the latent space of a pretrained graph VAE and uses separate guidance scales for the target signature and structural anchoring to a seed molecule. The method is evaluated on logP optimization, docking-score optimization, transcriptomic-guided generation, and Cell-Painting-guided generation.
Critical strengths
-
The problem formulation is interesting and practically relevant. Combining phenotype-conditioned generation with hit-to-lead-style molecular editing is a useful setting, and the separation of target and structural guidance is intuitive.
-
The logP experiments provide useful evidence for the proposed guidance mechanism. The comparison with standard CFG and the additional ablations suggest that c-CFG provides useful control over target optimization and structural preservation.
Critical weaknesses
-
The main biological editing setting is not evaluated as directly as the logP task. The transcriptomic experiment uses seed molecules, but target relevance is mainly measured by similarity to known ligands. The main Cell Painting experiment is de novo generation, while the editing results are mostly qualitative recovery examples. I think the paper demonstrates molecular editing clearly on logP and biological conditioning clearly on the transcriptomic and Cell Painting tasks, but the combination of the two is less directly quantified.
-
Some key experiments do not clearly isolate the source of the reported gains. The docking experiments claim a new SOTA while omitting post-MOOD methods such as GEAM [1] and f-RAG [2], which report stronger results on the same five-target docking benchmark. In the transcriptomic experiment, retrieval and editing are evaluated together, making it difficult to tell how much each contributes. The curriculum ablation also changes several training factors at once, so it is unclear whether the improvement comes from the curriculum itself.
Questions for authors
-
Can the authors provide quantitative results for the editing setting illustrated in Figure 1 on either the transcriptomic or Cell Painting task, reporting both target relevance and similarity to the input seed?
-
What do the retrieved transcriptomic seeds already achieve before editing? How much improvement does PhAME provide over these seeds across the full set of targets?
-
Why are post-MOOD docking methods such as GEAM [1] and f-RAG [2] omitted from Table 2 despite the paper's SOTA claim? Both report stronger results on the same five-target docking benchmark.
-
Does the curriculum still improve docking performance when the training budget is controlled? Also, which percentile schedule was actually used in Tables 8–9?
-
What evidence supports the claims about MoA-aware chemistry and structure–activity relationships given that the Cell Painting evaluation is based on structural proxies? Could the authors also clarify the statement that only PhAME exceeds chance in all three representation spaces? The GFlowNet means in Table 11 appear to exceed the 14.3% chance level in all three spaces.
References:
[1] Lee et al., Drug Discovery with Dynamic Goal-aware Fragments, ICML 2024.
[2] Lee et al., Molecule Generation with Fragment Retrieval Augmentation, NeurIPS 2024.
6 comments
Q4: Curriculum ablation
At a matched training budget, curriculum learning achieves the best mean hit rate on four of five targets and the best mean docking score on three of five targets. We compared it with two Stage 1-only baselines: No Curriculum preserves the original learning-rate schedule and optimizer resets, whereas Extended Stage 1 continues training at the Stage 1 learning rate without resetting AdamW. All variants use the same total number of optimizer updates and batch size. The gains are particularly pronounced for FA7 and BRAF. For FA7, curriculum increases the mean hit rate from 3.14-3.29% to 6.37%, approximately doubling performance. For BRAF, it increases the hit rate from 3.14-3.52% to 6.10%.
Docking Score – lower is better
| Target | Curriculum | No Curriculum | Extended Stage 1 |
|---|---|---|---|
| PARP1 | −11.234 0.077 | −11.423 0.066 | −11.393 0.066 |
| FA7 | −9.143 0.058 | −8.800 0.100 | −8.754 0.174 |
| 5HT1B | −11.270 0.140 | −11.163 0.184 | −11.155 0.131 |
| JAK2 | −10.433 0.109 | −10.425 0.072 | −10.482 0.152 |
| BRAF | −11.089 0.144 | −10.637 0.392 | −10.606 0.266 |
Hit Rate (%) – higher is better
| Target | Curriculum | No Curriculum | Extended Stage 1 |
|---|---|---|---|
| PARP1 | 11.27 0.84 | 13.67 0.78 | 14.04 0.99 |
| FA7 | 6.37 0.65 | 3.29 0.45 | 3.14 0.87 |
| 5HT1B | 13.58 1.96 | 12.93 2.26 | 13.15 3.14 |
| JAK2 | 11.89 1.12 | 11.49 2.06 | 11.30 2.55 |
| BRAF | 6.10 1.16 | 3.52 1.16 | 3.14 1.34 |
We apologize for the inconsistent reporting. The experiments underlying Tables 8–9 used the 50th → 99th → 99.8th percentile schedule for all five targets. The reference to the 90th percentile in Appendix D was incorrect, as were the FA7 and BRAF threshold values reported in Table 9. We have corrected these entries and clarified the actual schedule in the revised manuscript.
Q5. Scope of MoA claims and clarification of chance baselines
Thank you for highlighting these points. PhAME achieves the best or tied-best performance in 20 of the 21 combinations of metrics and representation spaces in Table 11 (in revised Table 17). This consistency across ECFP, CLOOME, and InfoAlign supports the robustness of the computational results and suggests that the gains are not specific to a single representation. It does not, however, directly establish the biological effects of generated molecules.
Previous work has demonstrated supervised MoA prediction from experimentally measured Cell Painting profiles extracted with CellProfiler (Way, Gregory P., et al). These approaches require morphological measurements for the compounds being assessed. Such measurements are unavailable for our novel generated molecules, and obtaining them would require new Cell Painting experiments. As detailed in GR1 of our General Response, the added Cell Painting evaluation uses measured profiles for recovered library compounds, providing phenotype-level evidence for that subset of generations. We have therefore revised the framing of these claims to reflect the scope of the evidence. Specifically, we have replaced the statements that the gains “reflect genuine MoA-aware chemistry” and demonstrate “a true understanding of structure-activity relationships” with statements that “the results suggest that the gains are not specific to a single embedding” and “the improvements suggest robustness to the choice of evaluation representation.” This distinguishes consistency across computational evaluations from experimental confirmation of biological activity. The Reviewer is also correct that GFlowNet’s mean Top-1 Cluster Accuracy exceeds the 14.3% chance level in all three representation spaces, reaching 21.2% in CLOOME, 17.1% in InfoAlign, and 17.0% in ECFP. Our original statement was incorrect when interpreted as a claim about Top-1 Cluster Accuracy alone, and we have corrected it.
The distinction supported by Table 11 (in revised Table 17) concerns performance across all evaluated metrics and representation spaces. For example, GFlowNet’s mean kNN Accuracy @1 in ECFP is 12.7%, below the corresponding chance baseline of 14.3%. The chance baseline is (1/7) for Top-1 Cluster Accuracy and kNN classification accuracy. We have therefore revised the statement to “Among the evaluated methods, PhAME is the only method whose reported mean exceeds the corresponding chance baseline for every evaluated metric in all three representation spaces.”
Way, Gregory P., et al. "Morphology and gene expression profiling provide complementary information for mapping cell state." Cell systems 13.11 (2022): 911-923.
Response to the rebuttal
Thanks for the detailed response and the additional experiments. The new docking comparisons and the matched-budget curriculum ablation address my concerns about those experiments. The Cell Painting results also make the trade-off clearer. I’m still unsure how often joint guidance helps with both editing goals on the same request, so I’ll keep my overall rating.
I have one question about the existing Cell Painting results. Among requests that could be scored under both joint guidance and no guidance, how often did joint guidance improve phenotype specificity without reducing similarity to the starting molecule?
W1/Q1. Quantitative evaluation of biological molecular editing.
We thank the Reviewer for highlighting the need for a more direct quantitative evaluation of biological molecular editing. We agree that the original manuscript demonstrated editing more directly for logP than for the biological tasks. To address this gap, we added a Cell Painting editing benchmark and expanded the transcriptomic editing evaluation, jointly reporting target relevance, similarity to the input seed, and molecular validity. These additions are summarized in GR1 and GR2 of our General Response, with full results in Appendices G and E.
W2/Q3. Docking comparisons under oracle-free and oracle-guided settings.
We thank the Reviewer for highlighting this omission. An important distinction, also emphasized in MOOD (Appendix E.2, Property Optimization), concerns the use of expensive oracle calls during generation. PhAME learns from a fixed, pre-scored dataset and subsequently generates candidates through classifier-free-guided denoising. Like MOOD, PhAME does not require docking-oracle calls during sampling. Unlike MOOD, PhAME also avoids gradients from a separately trained property predictor, using classifier-free guidance instead. GEAM and f-RAG use target-property feedback to adapt their search. Their stronger docking scores are obtained in a different generation regime, which should be made explicit in the comparison. Nevertheless, this distinction does not justify an unrestricted docking SOTA claim, particularly because our table already includes oracle-guided baselines: REINVENT, MORLD, FREED, FREED-QS.
Following the Reviewer’s comment, we have divided the results in Table 3 into oracle-free and oracle-guided generation methods. Among the evaluated oracle-free methods, PhAME achieves the best mean docking score on all five targets and the highest mean hit ratio on four of the five targets.
For oracle-guided generation, we use the baseline results reported in GEAM and additionally compare with f-RAG. We evaluate PhAME in an editing setting with a lightweight random forest surrogate that chooses candidates for the next optimization round. PhAME + RF achieves competitive results compared with GEAM and f-RAG. It achieves the highest hit ratio on all five targets, and the best mean docking score on FA7, while falling behind f-RAG on the remaining targets and remaining close to GEAM.
We have also compared computational runtimes. The PhAME + RF pipeline produces and docks 3,000 selected candidates in less than two hours on an RTX 4090, including surrogate-model training, PhAME training, molecule generation, and docking. In our tests, GEAM required approximately 2.5 hours for the corresponding task, while f-RAG reports approximately two hours per docking task in its appendix.
For transcriptomic-guided editing, please see the “Transcriptomic-guided editing” subsection of GR2 in our General Response.
For the curriculum ablation, please also see our response to Q4.
Q2. Improvement over transcriptomic retrieval.
PhAME improves the maximum Tanimoto similarity to known binders over the retrieved seeds by 5.3%-73.0% across targets (+30.4% on average).