- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper formulates "actionable" molecular optimization: each generated molecule must be reachable from an initial set via matched-molecular-pair (MMP) paths. MolWorld grows a molecule-transfer graph. A "molecule world model" (JEPA-style graph encoder, Transformer dynamics model predicting the latent embedding of an expansion, property-specific ranking scorer) selects five-molecule anchor contexts. A T5Chem-based set-conditioned generator proposes candidates, and RDKit MMP checks admit them. Evaluation covers PMO-style optimization (GSK3β/QED/JNK3/DRD2; ZINC250 10k calls, PMV21 1k calls) and Vina docking (1k calls), with top-k AUC over reachable molecules and an isolated-node ratio (Iso).
Critical strengths
-
A new, practically motivated formulation with a system that natively exploits analogue-series structure. Ablations show the pieces matter (Fixed Search Graph −9.1/−11.7 AUC points; STONED generator 22.2 vs 55.2 AUC@100; Fig. 4).
-
Unusually transparent evaluation: 9 baselines, 3 seeds with SDs, a random-pool control, encoder/dynamics validation (edge removal raises masked-node loss 3.45×; dynamics error −76.6% vs a mean target), and self-initiated leakage analyses (Tables 8–9) that report unfavourable results.
Critical weaknesses
- The comparison is not information- or constraint-matched.
- (a) The property scorer is trained on oracle labels over ChEMBL-derived expansions (84,490 outcomes) outside the call budget. The generator is also ChEMBL-trained, which likely overlaps the public bioactivity data behind the TDC activity oracles. On ZINC250 JNK3/DRD2, 50%/39% of generated molecules (20/30 and 18/30 top-10 entries) have scaffolds in the scorer's training set but not in the initial graph (Table 9). How much of the margin this explains is not quantified.
- (b) Seeding uses "cached" property values of the initial graph (up to ~10^5 molecules). It is unclear whether these are budgeted or given to baselines.
- (c) Reachability is built into MolWorld, while baselines are unconstrained and post-filtered. Fig. 7 (all valid molecules, not reachable-only) shows several baselines approaching MolWorld by 10k calls and everything saturating on QED. Table 1's margin therefore largely reflects the reachability filter. MMP-aware baselines (e.g. the cited MMP-conditioned transformers, or GraphGA/MolLEO with MMP-compatible operators) and unconstrained PMO numbers are missing.
- (d) In Fig. 4, Random Selection (57.3/51.3 AUC@10/100) already exceeds the best PMV21 baseline means I compute from Table 1 (≈55.3 GenMol, ≈33 MolLEO). The generator and graph design, not the world model, explain most of the gain. The STONED ablation also conflates quality with reachability, since random SELFIES mutations are rarely MMP-compatible.
- "Actionability" and the "world model" are weakly validated.
- Actionability means an RDKit MMP (≤3 cuts, variable fragment ≤33% of heavy atoms) to any node of a graph of hundreds to ~10^5 molecules. That is a loose criterion; the Fig. 6 text notes G4 differs from G1 at several positions.
- There is no hop-distance distribution, per-edge property deltas, SA, or chemist assessment. The QED case study is descriptor-level.
- Figs. 3/5/8–10 show counts or trends without baselines, and counts favour whoever emits more reachable molecules. PMV21 GSK3β/JNK3 scores stay ≤~0.5/0.35, so "high-property" is overstated.
- The world-model-specific gain over a Direct Context Ranker is ~+3 (heuristic pool) to ~+4 (random pool) AUC points. It has no error bars, uses one dataset (PMV21, 1k calls), and the y-axis starts at 20. No code release is mentioned.
Questions for authors
- How many oracle-labelled molecules train each scorer? How does performance change with a scorer trained without test-property labels, or only online?
- Are initial-graph property values free? Did every baseline receive the same initial set and values?
Suggestion:
- Report hop distance from the initial set for top-10/100 molecules, the fraction reachable only via generated intermediates, and per-edge property deltas.
- Add MMP-aware baselines, unconstrained PMO results, ablation SDs on ZINC250, diversity/novelty/SA, and a held-out oracle.
2 comments
W2. We have added the analyses the reviewer asks for.
Hop distance and dependence on generated intermediates. Generated molecules were filtered by the batch-wise reachability rule, ranked by property, and their shortest distance to the initial set measured on the final graph. Mean ± s.d. across three seeds:
| Property | Top-10 hops | Top-10 via generated intermediates | Top-100 via generated intermediates |
|---|---|---|---|
| QED | 2.77 ± 0.29 | 96.67% ± 5.77% | 94.67% ± 5.86% |
| JNK3 | 2.87 ± 0.81 | 100% ± 0% | 94.00% ± 6.56% |
| DRD2 | 2.43 ± 0.35 | 100% ± 0% | 99.67% ± 0.58% |
| GSK3β | 3.53 ± 0.57 | 100% ± 0% | 100% ± 0% |
Across all runs, 119 of 120 Top-10 molecules (99.17%) have no MMP edge to the initial set and are reachable only through molecules the search itself produced; for Top-100 the figure is 1165 of 1200 (97.08%). The paths are forward in time rather than completed in hindsight — one Top-10 molecule on GSK3β seed 29 sits six steps out, its nodes entering at rounds 0, 6, 7, 10, 12, 13 and 14.
Per-edge property deltas. We measured the change along each edge in the direction generation takes, from an initial-graph molecule to a new molecule attached to it, over all 1,000 generated molecules per run and across three seeds:
| Property | Initial | Generated | Change | Edges improving |
|---|---|---|---|---|
| QED | 0.6094 | 0.7785 | +0.1691 ± 0.0038 | 83.98% |
| JNK3 | 0.0649 | 0.0982 | +0.0332 ± 0.0233 | 63.48% |
| DRD2 | 0.5160 | 0.7818 | +0.2658 ± 0.0360 | 86.61% |
| GSK3β | 0.0822 | 0.1609 | +0.0787 ± 0.0066 | 73.58% |
On every objective the property improves across the majority of these edges, so the local modifications are moving in a consistent direction rather than drifting.
Synthetic accessibility. SA scores for the reachable generated molecules, against the 722 molecules of the initial graph (3.19 ± 0.29):
| Property | All reachable | Top-10 | Top-100 |
|---|---|---|---|
| QED | 2.88 ± 0.36 | 2.70 ± 0.34 | 2.73 ± 0.33 |
| JNK3 | 3.13 ± 0.25 | 3.19 ± 0.33 | 3.17 ± 0.30 |
| DRD2 | 2.85 ± 0.24 | 2.80 ± 0.15 | 2.82 ± 0.21 |
| GSK3β | 3.40 ± 0.28 | 3.49 ± 0.29 | 3.60 ± 0.26 |
Generated molecules stay within the synthetic accessibility of the series they come from, on QED and DRD2 somewhat below it, and neither the Top-10 nor the Top-100 sets drift away from it.
We thank the reviewer again for the detailed suggestions, and hope these additions address the concerns raised.
We thank the reviewer for the careful reading and for the generous assessment of the formulation and of our transparency.
W1. (a) & (b), Q1 & Q2. Each property scorer is trained on labelling that covers roughly 355k molecules. Neither this labelling nor the property values of the initial graph counts against the reported budget, but all baselines were given the same initial set and the same precomputed scores. On docking we do train the scorer online only, since the oracle is expensive there, and the initial graph carries docking scores for just 276 molecules. MolWorld still has the best AUC@10 and AUC@100 on all three targets (Table 2).
How much of the margin the scaffold overlap explains is answered most directly by PMV21, where the initial graph shares no scaffolds with the scorer's label training set and generated-molecule overlap is zero on GSK3β, JNK3 and DRD2, Top-10 and Top-100 included. MolWorld attains the highest AUC@10 and AUC@100 on all three regardless, so the margin survives where there is no overlap to explain it. ZINC250 admits a direct measurement, reported in Table 8. Zeroing the score contributions of overlapping molecules and recomputing moves mean AUC@10 by at most 0.000469 and mean AUC@100 by at most 0.000656.
(c). We have added an opponent that satisfies reachability natively, searching MMP transformation space directly under the same initial graphs, reachability rules and oracle budget. Each round draws a batch of molecules from the current graph, at random or greedily by property, and applies several random valid MMP transformations to each, so every candidate is by construction an MMP of a molecule already in the graph and Iso is exactly zero.
AUC@10 on PMV21. The greedy variant already beats every method in Table 7 yet still falls short of MolWorld, and the gap widens in AUC@100, from 0.6701 to 0.8194 on QED and from 0.6326 to 0.8205 on DRD2. With reachability identical on both sides, what remains can only come from whether the search has direction.
(d). We have added the anchor-only dynamics control the reviewer's Q2 asks for, removing the current-graph representation so that the expansion is predicted from the context alone, everything else held fixed.
Averaged over the four properties on PMV21, three seeds. Dropping the graph state costs most of the gain, which locates it in conditioning the expansion prediction on the current state rather than in the extra stage as such. Standard deviations for all ablation variants have been added to Figure 4.
We agree that Random Selection exceeding the baselines points at the framework rather than the world model, and this is what contributions 1 and 3 claim — the reachability formulation, the context-conditioned generator and the iterative graph expansion account for most of the gain, with the world model adding roughly three points on top. The four selection variants are worth reading together in this light. Random at 0.5727, Mean-Property at 0.5809, Direct Ranker at 0.5889 and anchor-only at 0.5885 fall within a band of under two points, while the full model sits clearly outside it, and the randomly sampled pool reproduces the same structure with 0.4545, 0.4897 and 0.4884 against MolWorld's 0.5280.
The reviewer is also right that STONED conflates quality with reachability. We have added a control that changes only the edges, running a single round over a fixed candidate context pool with the context molecules, backbone and training protocol held constant.
With the selected molecules identical, altering how the MMP relations are presented is enough to lower Top-10, so the generator is drawing on them.