- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces CLEF to predict EI-MS spectra from molecular graphs, enumerated bond-cleavage events, and candidate fragment formulas. Composition-matched cross-attention updates formula tokens with cleavage context to guide atom pooling and isotope rendering. The model trains end-to-end on structure-spectrum pairs without fragment-level annotations. Evaluation covers spectrum prediction, isomer retrieval, and cross-library transfer on NIST23, MoNA, and SWGDRUG. CLEF outperforms the baselines across these tasks by conditioning formula scoring on local cleavage context.
Critical strengths
Conditioning formula scores on candidate bond cleavages introduces a structural bias that separates constitutional isomers sharing fragment formulas. CLEF shows improvements across spectral and retrieval metrics. Strong zero-shot transfer from MoNA to SWGDRUG shows that predictions generalize across libraries. Scaffold and anonymous topology splits reduce structural leakage, while shared PubChem decoys standardize isomer retrieval.
Critical weaknesses
Methodological novelty is limited: formula scoring and fragmentation graphs are established, and the retrieval setup follows SCARF without initial attribution.
The ablations do not isolate composition-matched attention from raw event features, additional model capacity, and latent-set processing.
MoNA’s same-source retrieval covers only 6-8 formula groups per split, leaving uncertainty about small performance gains. The data-efficiency experiments also retain the full labeled validation set across all training fractions.
Questions for authors
Can the authors explain the retrieval protocol of SCARF and clarify the methodological advance over the existing fragmentation models?
Does composition-matched attention outperform simple aggregation of matching events or unrestricted event attention under comparable parameter and training budgets?
Can the authors report paired, formula-group-aware confidence intervals for retrieval and evaluate isomer retrieval on the external SWGDRUG cohort?
Do the limited-data gains hold with a proportionally reduced validation set, and what are the preprocessing times, inference throughput, and filter coverage losses?
4 comments
Q3
Q3. Formula-group-aware uncertainty and external isomer retrieval.
For NIST23 cross-source retrieval, we provide paired formula-group bootstrap estimates here. After averaging query-level differences across three training runs, we resample formula groups with replacement, retain all queries in each selected group, and recompute the query-weighted difference using the same draw for both models. The cohort contains 10,156 queries in 3,527 groups. Over 10,000 resamples, the R@1 difference is 0.0413 with a 95 percent interval of , and the MRR difference is 0.0311 with interval . These estimates concern NIST23. MoNA same-source retrieval retains 6 to 8 formula groups per split.
Table 2 adds all four retrieval metrics on SWGDRUG using MoNA-trained models. CLEF has same-source R@1/MRR of 0.209/0.463 and cross-source R@1/MRR of 0.270/0.427. RASSP:SN has 0.197/0.453 and 0.143/0.289. MassFormer has the highest same-source R@1, 0.337. CLEF has the highest cross-source R@1 and MRR among the transferred trainable models.
Table 2. SWGDRUG spectrum and isomer-retrieval evaluation. Trainable models transfer from MoNA, with mean sample SD. Bold and underline denote the best and second-best means among matched methods. CFM-ID () is a frozen external reference. Details are in Appendix A.2.
\hline & & \text{Spectrum prediction} & & & \text{Same-source (}K{=}5\text{)} & & \text{Cross-source (}K{=}50\text{)} & \\\\ \text{Method} & \text{DP }\uparrow & \text{SDP }\uparrow & \text{COS }\uparrow & \text{Top_1 }\uparrow & \text{R@1 }\uparrow & \text{MRR }\uparrow & \text{R@1 }\uparrow & \text{MRR }\uparrow \\\\ \hline \text{CLEF} & \mathbf{0.724}\pm0.007 & \mathbf{0.651}\pm0.013 & \mathbf{0.605}\pm0.018 & \mathbf{0.413}\pm0.027 & 0.209\pm0.028 & 0.463\pm0.014 & \mathbf{0.270}\pm0.021 & \mathbf{0.427}\pm0.017 \\\\ \text{RASSP:SN} & \underline{0.648}\pm0.005 & \underline{0.569}\pm0.009 & \underline{0.439}\pm0.015 & \underline{0.212}\pm0.018 & 0.197\pm0.040 & 0.453\pm0.022 & \underline{0.143}\pm0.009 & \underline{0.289}\pm0.009 \\\\ \text{MassFormer} & 0.561\pm0.006 & 0.498\pm0.006 & 0.331\pm0.007 & 0.150\pm0.011 & \mathbf{0.337}\pm0.022 & \underline{0.572}\pm0.015 & 0.079\pm0.010 & 0.194\pm0.008 \\\\ \text{RASSP:FN} & 0.546\pm0.018 & 0.432\pm0.034 & 0.297\pm0.013 & 0.128\pm0.009 & 0.185\pm0.032 & 0.448\pm0.019 & 0.054\pm0.006 & 0.149\pm0.008 \\\\ \text{MolFormer} & 0.478\pm0.010 & 0.506\pm0.004 & 0.308\pm0.008 & 0.127\pm0.010 & 0.316\pm0.034 & 0.552\pm0.025 & 0.072\pm0.006 & 0.182\pm0.006 \\\\ \text{NEIMS} & 0.568\pm0.006 & 0.517\pm0.006 & 0.354\pm0.006 & 0.160\pm0.005 & \underline{0.332}\pm0.025 & \mathbf{0.577}\pm0.021 & 0.072\pm0.007 & 0.181\pm0.007 \\\\ \text{Hybrid} & 0.385\pm0.018 & 0.493\pm0.009 & 0.155\pm0.021 & 0.040\pm0.022 & 0.205\pm0.026 & 0.457\pm0.014 & 0.063\pm0.007 & 0.172\pm0.008 \\\\ \text{CFM-ID}^{\dagger} & 0.552\pm0.000 & 0.500\pm0.001 & 0.534\pm0.000 & 0.338\pm0.001 & \text{–} & \text{–} & \text{–} & \text{–} \\\\ \hline \end{array}$$Q4.1
Q4. Reduced validation, preprocessing, throughput and filtering coverage.
We reselect checkpoints on nested validation subsets scaled with training fraction and evaluate on the corresponding nested test subsets (Appendix C.1, Figure 5). CLEF has higher means than RASSP:SN on all eight metrics at each fraction.
Figure 5 data. Each cell gives mean ± sample SD in CLEF / RASSP:SN order. Bold marks the higher mean. The 100% row uses the full validation and test sets.
Spectrum prediction
\hline \text{Train} & \text{DP} & \text{SDP} & \text{COS} & \text{Top_1} \\\\ \hline 1\\% & \mathbf{0.7389}\pm0.0135\\,/\\,0.6769\pm0.0159 & \mathbf{0.7420}\pm0.0330\\,/\\,0.6880\pm0.0350 & \mathbf{0.5930}\pm0.0232\\,/\\,0.4785\pm0.0304 & \mathbf{0.3147}\pm0.0196\\,/\\,0.1825\pm0.0465 \\\\ 5\\% & \mathbf{0.7952}\pm0.0018\\,/\\,0.7475\pm0.0028 & \mathbf{0.7919}\pm0.0065\\,/\\,0.7505\pm0.0117 & \mathbf{0.6875}\pm0.0008\\,/\\,0.6106\pm0.0040 & \mathbf{0.4210}\pm0.0087\\,/\\,0.3214\pm0.0062 \\\\ 10\\% & \mathbf{0.8209}\pm0.0015\\,/\\,0.7687\pm0.0029 & \mathbf{0.8197}\pm0.0039\\,/\\,0.7777\pm0.0090 & \mathbf{0.7243}\pm0.0044\\,/\\,0.6487\pm0.0082 & \mathbf{0.4582}\pm0.0162\\,/\\,0.3622\pm0.0123 \\\\ 25\\% & \mathbf{0.8488}\pm0.0010\\,/\\,0.8028\pm0.0005 & \mathbf{0.8479}\pm0.0037\\,/\\,0.8111\pm0.0032 & \mathbf{0.7723}\pm0.0028\\,/\\,0.7129\pm0.0017 & \mathbf{0.5181}\pm0.0046\\,/\\,0.4372\pm0.0020 \\\\ 100\\% & \mathbf{0.8743}\pm0.0015\\,/\\,0.8388\pm0.0006 & \mathbf{0.8766}\pm0.0011\\,/\\,0.8514\pm0.0015 & \mathbf{0.8174}\pm0.0026\\,/\\,0.7783\pm0.0015 & \mathbf{0.5832}\pm0.0048\\,/\\,0.5287\pm0.0033 \\\\ \hline \end{array}$$ **Isomer retrieval** $$\begin{array}{crrrr} \hline \text{Train} & \text{K=5 R@1} & \text{K=5 MRR} & \text{K=50 R@1} & \text{K=50 MRR} \\\\ \hline 1\\% & \mathbf{0.4815}\pm0.0602\\,/\\,0.3998\pm0.0351 & \mathbf{0.6764}\pm0.0379\\,/\\,0.6233\pm0.0144 & \mathbf{0.2594}\pm0.0281\\,/\\,0.1440\pm0.0556 & \mathbf{0.3932}\pm0.0266\\,/\\,0.2711\pm0.0303 \\\\ 5\\% & \mathbf{0.5596}\pm0.0090\\,/\\,0.5420\pm0.0072 & \mathbf{0.7313}\pm0.0078\\,/\\,0.7163\pm0.0019 & \mathbf{0.3306}\pm0.0083\\,/\\,0.3051\pm0.0115 & \mathbf{0.4763}\pm0.0097\\,/\\,0.4425\pm0.0093 \\\\ 10\\% & \mathbf{0.6089}\pm0.0065\\,/\\,0.5692\pm0.0150 & \mathbf{0.7622}\pm0.0060\\,/\\,0.7383\pm0.0092 & \mathbf{0.3948}\pm0.0050\\,/\\,0.3236\pm0.0105 & \mathbf{0.5358}\pm0.0045\\,/\\,0.4719\pm0.0080 \\\\ 25\\% & \mathbf{0.6545}\pm0.0112\\,/\\,0.6274\pm0.0041 & \mathbf{0.7935}\pm0.0057\\,/\\,0.7754\pm0.0026 & \mathbf{0.4673}\pm0.0047\\,/\\,0.4145\pm0.0154 & \mathbf{0.5976}\pm0.0064\\,/\\,0.5548\pm0.0100 \\\\ 100\\% & \mathbf{0.7192}\pm0.0050\\,/\\,0.6978\pm0.0022 & \mathbf{0.8353}\pm0.0030\\,/\\,0.8223\pm0.0014 & \mathbf{0.5596}\pm0.0036\\,/\\,0.5183\pm0.0044 & \mathbf{0.6787}\pm0.0025\\,/\\,0.6477\pm0.0032 \\\\ \hline \end{array}$$ **Table 5.** Inference costs on an NVIDIA RTX 4090 for 1,000 NIST23 test structures. $$\begin{array}{lrrrrr}\hline\text{Method}&\text{Params (M)}&\text{Latency (ms)}&\text{Batch}&\text{Throughput (mol/s)}&\text{GPU memory (MiB)}\\\\\hline\text{CLEF}&26.4&25.4&8&132.2&8245.6\\\\\text{RASSP:SN}&26.3&13.2&4&150.5&1147.3\\\\\text{MassFormer}&54.6&18.3&120&171.8&566.9\\\\\text{RASSP:FN}&19.7&13.8&8&181.6&8129.1\\\\\text{MolFormer}&58.8&38.7&120&1564.5&353.1\\\\\text{NEIMS}&39.3&2.9&512&25179.2&250.3\\\\\text{Hybrid}&2.9&4.8&128&16661.6&424.8\\\\\hline\end{array}$$ Latency is measured at batch size 1; throughput uses the listed batch size. CLEF uses cached event and formula features. **Table 7.** Stepwise MoNA selection. The first nine rows count spectrum records and the final row counts consensus targets. The last column reports records removed or collapsed at that step. $$\begin{array}{lrr}\hline\text{Stage}&\text{Retained}&\text{Removed or collapsed}\\\\\hline\text{GC–MS source export}&19,136&\text{–}\\\\\text{EI, MS1 and positive-mode metadata}&16,672&2,464\\\\\text{Exclude explicit non-70 eV energy}&13,799&2,873\\\\\text{Exclude derivatization or mixture annotations}&11,205&2,594\\\\\text{Accepted license metadata}&7,844&3,361\\\\\text{Structure, component, isotope and element checks}&5,729&2,115\\\\\text{Capacity limits: N48/M400/F8192}&5,308&421\\\\\text{Spectrum quality control}&5,280&28\\\\\text{Same-identity exact-spectrum deduplication}&5,265&15\\\\\text{Consensus target construction}&3,977&1,288\\\\\hline\end{array}$$Q4.2
Q4 (continued). MoNA filtering coverage, relaxed selection and stratification.
Appendix A.3, Tables 7 and 8, reports stepwise exclusions from the 19,136-record GC-MS export to 5,265 retained spectra and 3,977 consensus targets. Tables 9 and 10 give the relaxed-cohort splits and exclusions. Shared computation and stepwise-coverage revisions are dark red.
Table 8. Record-level exclusion accounting. Capacity exclusions are assigned to the first failed rule.
\hline \text{Stage} & \text{Exclusion rule} & \text{Records} \\\\ \hline \text{Acquisition metadata} & \text{CI-B / generic GC–MS / missing instrument / MS2 / negative mode / other} & \text{992 / 918 / 390 / 143 / 8 / 13} \\\\ \text{Structure checks} & \text{SMILES–InChI conflict} & 1,298 \\\\ & \text{Header identity mismatch} & 1 \\\\ & \text{Multiple components} & 5 \\\\ & \text{Isotope-labelled structures} & 133 \\\\ & \text{Unsupported elements} & 678 \\\\ \text{Capacity checks} & \text{Atom count / mass / formula count, sequential first failure} & \text{374 / 11 / 36} \\\\ \text{Spectrum checks} & \text{Nonpositive intensity} & 2 \\\\ & \text{Outside-grid total ion current} & 2 \\\\ & \text{Signal outside the precursor isotope envelope} & 24 \\\\ \text{Deduplication} & \text{Exact spectra within the same identity} & 15 \\\\ \hline \end{array}$$ Under both relaxed filters (N48/M400 and N96/M800), CLEF leads all four spectrum metrics and both $K=50$ retrieval metrics. RASSP:SN leads both $K=5$ metrics in N96/M800 (Table 11, dark orange). **Table 11.** Spectrum prediction and isomer retrieval under relaxed MoNA selection. $$\begin{array}{llrrrrrrrr}\hline&&&\text{Spectrum prediction}&&&K=5&&K=50&\\\\\text{Setting}&\text{Method}&\text{DP }\uparrow&\text{SDP }\uparrow&\text{COS }\uparrow&\text{Top_1 }\uparrow&\text{R@1 }\uparrow&\text{MRR }\uparrow&\text{R@1 }\uparrow&\text{MRR }\uparrow\\\\\hline\text{N48/M400}&\text{CLEF}&{\bf0.7940}&{\bf0.7814}&{\bf0.7315}&{\bf0.5066}&{\bf0.7477}&{\bf0.8480}&{\bf0.4848}&{\bf0.6054}\\\\&\text{RASSP:SN}&0.7619&0.7412&0.6810&0.4468&0.7451&0.8436&0.4560&0.5621\\\\&\text{MassFormer}&0.7464&0.7050&0.6219&0.3484&0.6432&0.7718&0.2704&0.4076\\\\\hline\text{N96/M800}&\text{CLEF}&{\bf0.7982}&{\bf0.7619}&{\bf0.7332}&{\bf0.4743}&0.7503&0.8620&{\bf0.4712}&{\bf0.6061}\\\\&\text{RASSP:SN}&0.7635&0.7367&0.6752&0.3653&{\bf0.7678}&{\bf0.8642}&0.4241&0.5509\\\\&\text{MassFormer}&0.7392&0.6938&0.5966&0.2731&0.6885&0.8082&0.3166&0.4427\\\\\hline\end{array}$$ On the original filtered cohort, CLEF has higher DP than RASSP:SN in all 17 strata (Table 13, dark orange). **Table 13.** MoNA molecular-property and energy-metadata strata. Record counts pool the three test splits, and scores are equal-cell means over nine split–training combinations. SN denotes RASSP:SN and MF denotes MassFormer. Bold marks the highest value among the three listed methods within each stratum. $$\begin{array}{llrrrrrrr}\hline&&&&\text{DP }\uparrow&&&\text{SDP }\uparrow&\\\\\text{Axis}&\text{Stratum}&\text{Records}&\text{CLEF}&\text{SN}&\text{MF}&\text{CLEF}&\text{SN}&\text{MF}\\\\\hline\text{Mass (Da)}&\lt 200&783&{\bf0.7927}&0.7749&0.7716&{\bf0.7601}&0.7394&0.7232\\\\&200\text{–}\lt 400&404&{\bf0.7342}&0.6854&0.6522&{\bf0.7244}&0.6767&0.6204\\\\\hline\text{Heavy atoms}&\leq10&392&{\bf0.7775}&0.7629&0.7654&{\bf0.7585}&0.7487&0.7462\\\\&\text{11–15}&528&{\bf0.8005}&0.7735&0.7589&{\bf0.7545}&0.7190&0.6830\\\\&\geq16&267&{\bf0.7133}&0.6618&0.6265&{\bf0.7185}&0.6707&0.6123\\\\\hline\text{Rings}&0&460&{\bf0.8558}&0.8261&0.8042&{\bf0.7778}&0.7185&0.6872\\\\&1&292&{\bf0.7051}&0.6921&0.6950&0.6629&0.6607&{\bf0.6644}\\\\&\geq2&435&{\bf0.7280}&0.6914&0.6738&{\bf0.7728}&0.7549&0.7058\\\\\hline\text{Rotatable bonds}&0&162&0.7478&0.7268&{\bf0.7674}&0.7802&{\bf0.7919}&0.7797\\\\&\text{1–3}&369&{\bf0.7364}&0.7155&0.7040&{\bf0.7599}&0.7524&0.7192\\\\&\geq4&656&{\bf0.7995}&0.7652&0.7372&{\bf0.7333}&0.6809&0.6485\\\\\hline\text{Formula count}&\leq255&226&0.7616&0.7452&{\bf0.7724}&0.7767&0.7816&{\bf0.7856}\\\\&\text{256–1,023}&676&{\bf0.8000}&0.7735&0.7619&{\bf0.7606}&0.7279&0.6988\\\\&\geq1{,}024&285&{\bf0.7177}&0.6754&0.6244&{\bf0.6964}&0.6457&0.5866\\\\\hline\text{Energy metadata}&\text{All report 70 eV}&1,071&{\bf0.7829}&0.7551&0.7419&{\bf0.7599}&0.7276&0.6984\\\\&\text{All unreported}&80&{\bf0.6438}&0.5924&0.5761&{\bf0.5811}&0.5543&0.5128\\\\&\text{Mixed member metadata}&36&{\bf0.7656}&0.7652&0.7543&0.7471&{\bf0.7771}&0.7472\\\\\hline\end{array}$$
Q1 Q2
Thank you for the detailed review. Revisions specific to your comments are purple. Shared methodological, computational and filtering revisions are dark red.
Q1. SCARF retrieval protocol and methodological advance.
SCARF (Goldman et al., 2023) ranks a target with 49 same-formula PubChem isomers selected by Morgan fingerprint Tanimoto similarity, using spectral cosine similarity. Section 2, Tandem prediction, now explains and cites this protocol. Section 4.1 attributes the candidate-construction strategy, which we apply to EI with candidate lists shared across methods.
Section 2 also distinguishes the computation. ICEBERG generates and scores fragmentation graphs. FraGNNet predicts fragment-node/formula probabilities and marginalizes over nodes. CLEF combines cleavage descriptors and local atom states through composition-matched cross-attention to update each formula representation before atom pooling and formula scoring (Sections 3.2 and 3.3, Figure 2). The revised discussion in Section 2 is dark red. The detailed definitions and diagram revisions are blue.
Q2. Uniform aggregation, unrestricted attention and model capacity.
We compare learned composition-matched attention with uniform aggregation and unrestricted all-event attention in Section 4.3 and Table 4(c). Uniform aggregation assigns equal weight to the events matching each formula. Section 4.3 also reports parameter-count and attention-complexity comparisons. Table 4(a,b) and Appendix C.2 provide the branch and descriptor controls and event-erasure analysis.
Table 4. NIST23 ablations. Bold marks the best value in each block.
\hline & & \text{Spectrum prediction} & & & \substack{\text{Same-source}\\\K=5} & & \substack{\text{Cross-source}\\\K=50} & \\\\ \text{Variant} & \text{DP} & \text{SDP} & \text{COS} & \text{Top_1} & \text{R@1} & \text{MRR} & \text{R@1} & \text{MRR} \\\\ \hline \rlap{\text{(a) Conditioning branches}} & & & & & & & & \\\\ \hline \text{Graph} & 0.7691 & 0.5051 & 0.6303 & 0.3463 & 0.4814 & 0.6806 & 0.1832 & 0.3134 \\\\ \text{Graph + formula} & 0.8091 & 0.7437 & 0.7001 & 0.4401 & 0.5492 & 0.7240 & 0.2845 & 0.4277 \\\\ \text{CLEF} & \mathbf{0.8729} & \mathbf{0.8758} & \mathbf{0.8147} & \mathbf{0.5782} & \mathbf{0.7386} & \mathbf{0.8477} & \mathbf{0.5567} & \mathbf{0.6762} \\\\ \hline \rlap{\text{(b) Event-descriptor alignment}} & & & & & & & & \\\\ \hline \text{CLEF} & \mathbf{0.8729} & \mathbf{0.8758} & \mathbf{0.8147} & \mathbf{0.5782} & \mathbf{0.7386} & \mathbf{0.8477} & \mathbf{0.5567} & \mathbf{0.6762} \\\\ \text{Shuffled descriptors} & 0.8577 & 0.8525 & 0.7887 & 0.5464 & 0.6945 & 0.8202 & 0.4824 & 0.6166 \\\\ \text{Formula–graph null} & 0.8072 & 0.7471 & 0.6965 & 0.4313 & 0.5483 & 0.7228 & 0.2920 & 0.4319 \\\\ \hline \rlap{\text{(c) Event aggregation}} & & & & & & & & \\\\ \hline \text{CLEF} & \mathbf{0.8729} & \mathbf{0.8758} & \mathbf{0.8147} & \mathbf{0.5782} & \mathbf{0.7386} & \mathbf{0.8477} & \mathbf{0.5567} & \mathbf{0.6762} \\\\ \text{Composition uniform} & 0.8700 & 0.8709 & 0.8097 & 0.5738 & 0.7293 & 0.8418 & 0.5432 & 0.6642 \\\\ \text{Full attention} & 0.7720 & 0.6535 & 0.6268 & 0.3466 & 0.4584 & 0.6619 & 0.1848 & 0.3102 \\\\ \hline \end{array}$$ K=5 denotes same-source retrieval and K=50 cross-source retrieval.