- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper tackles the lead optimization pipeline and aims to improve desired properties while maintaining structural similarity to the lead compound. The method uses a pre-trained SMILES encoder with frozen embeddings, whose latents are fed into SCIN network used to construct supervised local improvement pairs. A source-anchored flow matching (SAFM) model then trains a velocity network to transport from source latents towards these mined targets; it has an edit budget which truncates the integration path. SAFM is conditioned on source embedding and property labels, and guided using classifier-free guidance. At inference time a trained decoder is used to map latent representations back into SMILES, generating eight candidates of which the first passing domain validity gates is kept. A regressor is trained as a property oracle to predict target property.
Critical strengths
This paper tackles the lead optimization pipeline and aims to improve desired properties while maintaining structural similarity to the lead compound. The method uses a pre-trained SMILES encoder with frozen embeddings, whose latents are fed into SCIN network used to construct supervised local improvement pairs. A source-anchored flow matching (SAFM) model then trains a velocity network to transport from source latents towards these mined targets; it has an edit budget which truncates the integration path. SAFM is conditioned on source embedding and property labels, and guided using classifier-free guidance. At inference time a trained decoder is used to map latent representations back into SMILES, generating eight candidates of which the first passing domain validity gates is kept. A regressor is trained as a property oracle to predict target property.
Critical weaknesses
How come the actual mined supervision targets fails the evaluation criteria? The paper describes mining "real improvement neighbours" and "supervised local improvement trajectories," which implies these targets should be acceptable edits, but on ZINC they are not: table 1 SCIN target retrieval (M3) scores 0. This suggests the model is trained to move toward targets its own evaluation would fail which, in the worst case, means that the quoted SAFM result is gained by the model 'not' reaching the desired training target. Section 4.2.1. Lays out ‘non-negative’ (Ln 293) improvement as a criteria for faithful-improve rate. A paper with a central claim to improve molecular properties should not count zero change as improvement.
Figure 4 case study shows the deletion of a functionally important group which the authors also concede is a plausible pharmacophore loss. Morgan-Tanomoto similarity score is indifferent to which substructures are preserved so edits that might remove functionally critical groups can still pass faithfulness criteria. As detailed in the introduction, lead optimization requires edits that improve a property while also preserving functionality/affinity.
No limitations mentioned: A section that promises limitations and delivers none reads as either unaware or evasive. If the decoder construction quality is around 73% for ZINC250K (Table 9), how come Table 3 and Table 5 recover 1.00 identity on tanimoto and constraint passing criteria? Please clarify. The big difference in constraint passing (0.730 vs 0.616) for In-house SAFM with and without conditioning in Table 3, makes the Table 1’s exactly same (0.363) faithfulness scores look implausible. I’m also not fully bought on the faithful improve rate as a metric. ‘The paper’s central question: whether an editor can improve a molecule locally, rather than generate a different high-scoring molecule.’ Appendix H counters this as the rearrangements described there gives a much better Tanimoto similarity (0.93 vs 0.84) compared to single-residue substitutions, despite affecting every residue in the macrocycle. Classifier free guidance should be referenced to Ho and Salimans (2022). The paper contains inconsistencies on important terms e.g. the method is called SAFM throughout in the paper yet Figure 5 describes it as “SC-CFM (ours)”. The primary metric of evaluation is called “fidelity-improvement trade off” or ‘faithful-improvement rate’. Table 5 calls ‘de novo filter’ what Table 1 calls ‘generate-and-filter’.
Questions for authors
Please Report Faithful-Improve on ZINC under similarity floors of 0.6 - 0.8, and statistical paired test over the 15 cells for SAFM vs. latent interpolation. Please also give the per-criterion failure breakdown for SAFM, SCIN retrieval and latent interpolation on ZINC, and state exactly what the small-molecule constraint gate
Correct the Faithful-Improve metric. Re-report Faithful-Improve under strict \Deltau>0 and provide the distribution of \Delta u among passing edits.
Please write the limitations section.
Please provide an analysis of nearest-neighbor in the training set for accepted edits including the distribution of molecules tanimoto similarity to the train set.
4 comments
We appreciate this scrutiny and have acted on both parts of it. The old Faithful-Improve score was the intersection of three gates (, Tanimoto , and constraint passage), not a direct function of the marginal constraint-pass rate. The source-conditioned and no-source variants passed the individual gates on different subsets of molecules, and their three-way intersections happened to yield the same score of . Thus, the identical composite scores were possible despite the different marginal constraint-pass rates, but we agree that the aggregate obscured how each variant succeeded and failed.
The revised main text therefore no longer uses the composite as the primary result. Table 1 reports the strict % rate, mean , and mean Tanimoto separately, while the expanded appendix tables report the constraint-pass rate. This makes every dimension of the fidelity--improvement trade-off visible. Consistent with Table 1, among constraint-passing peptide edits, SAFM with source conditioning achieves a % strict improvement rate, mean of , and mean Tanimoto of ; without source conditioning, the corresponding values are %, , and . The constraint-pass rates over all decoded edits, reported separately in the expanded appendix tables, are % and %, respectively.
We agree. Morgan Tanimoto is a global measure, so an edit that conserves the residue multiset can score above a single-residue substitution while moving every residue, and the fingerprint floor alone does not establish locality.
In the re-run analysis, 156 of the 367 accepted edits invert a stereocentre and score exactly 1.00, the fingerprint being insensitive to configuration. Rearrangements reach 0.87 and 0.88 against 0.84 for single-residue substitutions, so the ordering holds with a narrower margin than the figures quoted.
Appendix C therefore classifies edits by residue multiset rather than by fingerprint. Under that classification the accepted edits are dominated by single-site changes, 241 of 367, with 51 rearrangements and 75 multi-residue changes.
We have moved the classification into Section 5.2 so that the locality claim rests on edit composition.
We have added the Ho & Salimans (2022) citation where classifier-free guidance is introduced in Sections 2 and 3.3. During this revision, we also aligned the sampler with this standard classifier-free-guidance formulation and re-ran the affected experiments under the unified protocol and strict criterion. The revised results are reported in Section 5.1 and summarised in the general response.
We thank the reviewer for identifying these inconsistencies. We have replaced "SC-CFM'' with "SAFM'' in the figure labels and legends, and standardised "de novo filter'' to "generate-and-filter'' throughout. We now reserve "fidelity-improvement trade-off'' for the relationship between property improvement and source similarity; the revised evaluation reports its components separately through the strict % rate, mean , mean Tanimoto, and constraint-pass rate rather than using Faithful-Improve as the primary composite metric. These revisions also address the metric concern raised in R3.3.
On ZINC the constraint gate requires only that the decoded SMILES is valid (RDKit-parseable) and distinct from the source by canonical-SMILES inequality. No additional structural or drug-likeness filter is applied. Figure 7 in Appendix B now reports the per-gate pass rates at across the 15 seed--fold cells.
We re-evaluated all methods on ZINC at Tanimoto floors from to across the 15 cells. SAFM without source conditioning leads latent interpolation at every floor up to : at floor it scores % against %; at , % against %; and at , % against %. At floor , the two converge at % and %, respectively. Generate-and-filter, SCIN retrieval, and SAFM with random targets fall below % by floor and reach % by floor .
A two-sided Wilcoxon signed-rank test over the 15 matched seed--fold cells confirms that SAFM (no source conditioning) significantly outperforms latent interpolation at all floors ( at each). At floor the difference remains significant (). At floor the two methods are statistically indistinguishable ().
We thank the reviewer for pressing this point. The revised manuscript enforces strict everywhere: in all result tables, and in the general response summary above. All methods were re-run under this tightened criterion with the same CV folds and seeds.
The revised headline results are reported in Table 1, with constraint-pass and per-gate results reported separately in Appendix B. This reporting exposes strict improvement, improvement magnitude, source similarity, and validity without collapsing them into a single primary score.
As an additional distributional view, we report among best per source at . On CycPeptMPDB (PAMPA), SAFM with source conditioning has , median , and IQR ; without source conditioning, , median , and IQR . On ZINC under analytic QED, SAFM with source conditioning has , median , and IQR ; without source conditioning, , median , and IQR .
Done. Section 6 of the revised manuscript now contains a dedicated "Discussion and Limitations'' subsection covering five points, as detailed in our response to R3.1 above: single-property optimisation, Tanimoto blind spots, absence of site-level control, proxy-based evaluation (with analytic QED on ZINC), and scope of the chemical characterisation.
For this analysis we selected the constraint-passing edits with positive improvement (), one per source at the best and budget , and computed the maximum Morgan-Tanimoto similarity (radius 2, 2048 bits, chirality not encoded) to any molecule in the corresponding training fold. On CycPeptMPDB (PAMPA, in-house oracle), SAFM with source conditioning () reaches mean nearest-neighbour Tanimoto and median , and without source conditioning () and .
A Tanimoto of here does not establish molecular identity, because the Morgan fingerprint is computed without stereochemistry. An edit that inverts a single stereocentre yields a diastereomer, an epimer with identical 2D connectivity and therefore an identical fingerprint, even though it is a distinct molecule. Section 5.2 attributes 156 of these unit-Tanimoto cases to stereocentre inversions. For cyclic peptides this distinction is chemically load-bearing, since backbone stereochemistry (L versus D residues) is among the primary determinants of passive membrane permeability, so an epimerization that the fingerprint cannot see is exactly the kind of permeability-relevant edit the model is meant to produce.
On ZINC, SAFM with and without source conditioning ( and ) reaches mean nearest-neighbour Tanimoto and , median and , with only % and % of edits above , so training-fold overlap is low. The contrast with the peptides partly reflects the size sensitivity of the fingerprint, since a comparable local modification changes a larger fraction of the substructural features in the smaller ZINC molecules than in the larger peptides. We discuss these limitations on Morgan Tanimoto under the Section 6.
Thank you for your detailed scrutiny of the supervision, evaluation and presentation.
Your central question concerns why mined improvement targets can fail the structural evaluation. SCIN mines pairs using latent cosine similarity and a property margin, whereas source retention is evaluated using Morgan-Tanimoto similarity; these criteria are distinct, particularly on ZINC, where mining imposes no additional structural constraint (Section 3.2). SCIN supplies observed improvement directions, and SAFM learns source-initialised transport with an edit budget that balances improvement and source retention rather than requiring reproduction of an individual mined target (Section 3.3). The revised evaluation requires strict positive improvement (): Table 1 reports its rate, improvement magnitude and source similarity separately, with constraint-pass rates in Appendix B. We also aligned the classifier-free-guidance sampler with Ho and Salimans (2022), added the citation and re-ran the affected comparisons under matched folds, seeds and evaluation conditions (Sections 2–3.3 and 5.1).
On your requested analyses: (1) at each ZINC similarity floor, reported percentages reflect distinctness from source, constraint pass, strict positive QED improvement and source similarity at or above the specific floor. At floor 0.5, SAFM without source conditioning succeeds for 24.3% of held-out sources versus 12.7% for latent interpolation. It significantly outperforms interpolation at floors 0.1–0.6 (p < 0.001 at each) and 0.7 (7.8% versus 6.6%; p = 0.033); at 0.8, the methods converge (2.62% versus 2.69%; p = 0.72) (added to Appendix B.3). (2) Appendix B now reports per-gate constraint breakdown for ZINC in addition to peptides, and Section 3.4 defines the ZINC gate as RDKit validity and canonical-SMILES distinctness. (3) Appendix B.4 now reports improvement distributions among constraint-passing edits, with positive medians for both SAFM variants on both datasets. (4) Appendix B.2 quantifies training-fold overlap through nearest-neighbour means and medians and explains why unit Tanimoto does not imply molecular identity when stereochemistry is omitted.
We have also resolved the reporting distinctions behind your questions: the Identity reference returned the source directly rather than reconstructing it through the decoder and has been removed. The earlier identical composite scores were intersections of gates on different subsets, not consequences of the marginal constraint-pass rates; the revised reporting exposes those components separately. On rearrangements, Morgan Tanimoto can rank a whole-ring residue rearrangement above a single-residue substitution, so we assess locality using complementary residue-level evidence rather than the fingerprint alone. This analysis identifies 241 single-site changes among 367 modifications in the analysed seed-0/fold-0 set across budgets (Section 5.2; Appendix C). On functionality, SAFM demonstrates controllable intrinsic-property improvement with source retention, providing an editing component for lead optimisation. Preserving binding affinity and biological function is explicitly discussed in Section 6 (Discussion and Limitations) as a direction for future target-aware, multi-property optimisation. We have elaborated on the limitations in Section 6, distinguishing source similarity from pharmacophore preservation and setting out the objective needed for full lead optimisation, alongside fingerprint blind spots, absence of site-level control, proxy-based evaluation and the scope of the chemical characterisation. Method and baseline terminology has been standardised.
Together, these corrections and analyses strengthen the evidence for controllable source-anchored editing while defining its scope precisely. We respectfully ask you to reconsider your assessment in light of the revised manuscript. As the discussion period draws to a close, please let us know if any point remains unclear or requires further clarification so that we can respond before it ends.
We thank the reviewer for recognising how SCIN, SAFM, budgeted inference, and decoding form an integrated source-anchored editing pipeline.
R1.1: SCIN targets versus evaluation criteria
We thank the reviewer for this observation. The key distinction is that SCIN targets provide directional supervision for the velocity field (they define where property improves relative to the source in latent space) and are not intended to be individually recoverable edit endpoints.
SCIN pairs are mined using latent cosine similarity (τ≥0.50) and a property margin (δmin), whereas the evaluation gate is applied in Morgan-fingerprint space (Tanimoto ≥0.70 to the source). These similarity measures are not equivalent: a higher-QED latent neighbour can remain distant from the source in fingerprint space. This mismatch is largest on ZINC, where g imposes no additional structural constraint and decoded constraint-passing SCIN retrieved candidates have mean source Tanimoto 0.09; on peptides, the equal-monomer-length restriction narrows the training-pair space and decoded constraint-passing SCIN retrieved candidates reach mean Tanimoto 0.56.
SAFM uses SCIN targets as directional supervision rather than endpoints to reproduce. At inference, it integrates a learned velocity field from the held-out source embedding without access to a target molecule, allowing partial transport to balance improvement and source retention. The random-pair ablation supports the contribution of SCIN neighbourhood structure on peptides: replacing SCIN with globally sampled improving pairs raises mean Δ from 0.33 to 0.48 among the constraint-passing edits, but reduces mean source Tanimoto from 0.85 to 0.60. Thus, broader supervision produces larger property gains at the cost of source retention, while SAFM need not retrieve an individual SCIN target for that supervision to be useful.
R1.2: Strict positive improvement criterion
Thank you for this valuable comment. We have substantially revised the manuscript to address this concern and enforce a strict improvement criterion (Δu>0). The updated numbers are reported in the revised Tables 1 and 3--5 and summarised above.
R2: Functional-group preservation and affinity
We agree. Morgan Tanimoto is indifferent to which fragments survive, so an edit can clear the 0.7 Tanimoto floor while removing a pharmacologically relevant group, as in Figure 4. The updated analysis adds a further case, since 156 of the 367 accepted edits invert a stereocentre and score a Tanimoto of exactly 1.00.
The new Limitations section states that the recipe improves an intrinsic property while leaving others unguarded, so lead optimisation in the full sense requires a target-aware objective. It also records the blind spots of the fingerprint floor that motivate reporting the residue-level classification alongside it.
R3.1: Missing limitations
Section 6 now carries a Limitations subsection, covering five points.
The recipe optimises one intrinsic property, so properties outside the objective are left unguarded, which the Figure 4 edit illustrates by removing a plausible pharmacophoric element. We set out the multi-property extension and the co-labelled data it would require.
Fidelity is measured by Morgan Tanimoto, which is global, indifferent to which fragments survive, and insensitive to configuration, so the 156 stereocentre inversions among the accepted peptide edits score exactly 1.00. The residue-level classification is reported alongside it for this reason.
SAFM acts on a pooled sequence embedding and decodes the whole molecule, so there is no site-level control, and the edit budget acts globally as the only inference-time dial.
For peptides, all reported property changes are predictions from learned proxies, and the cross-assay analysis in the appendix compares predictors rather than measurements. QED on ZINC is computed analytically via RDKit.
The chemical characterisation covers one assay on one dataset, and its conformational component is computed in vacuum with a classical force field.
R3.2: Identity baseline and decoder quality
We thank the reviewer for the keen observation. The Identity row (M0) is a sanity check: it returns the source molecule directly without passing through the decoder, so it trivially scores 1.00 on Tanimoto. It is not a claim about decoder quality. We agree that including it alongside decoder-dependent methods invites exactly this confusion, so we have removed the Identity row from the revised tables.