acceptodds
Withdrawn from ICLR 2027

From Static Ranking to Active Planning: Feedback-Conditioned Candidate Verification for Molecule Identification with Neural Spectrum Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Molecule identification with neural spectrum prediction ranks candidate molecular structures by agreement between their predicted spectra and observed experimental spectra, then verifies candidates in this order. However, simply following this static ranking overlooks spectral similarity feedback obtained during candidate verification. We introduce ActiveMS, an active planning framework for MS/MS-based molecule identification that incorporates this feedback to determine which unverified candidate structures to verify next. ActiveMS-GP models feedback residuals with a Gaussian process, while ActiveMS-FCP learns an amortized feedback-conditioned posterior over candidate structures across verification rollouts. For evaluation, we construct ActiveMS Bench from the SpectraVerse and NIST spectral libraries as two separate benchmarks, with feedback supplied by an independent non-neural simulator. On ActiveMS Bench, both planners consistently outperform static-ranking verification with MassFormer and ICEBERG spectrum predictors. With either predictor, ActiveMS-GP and ActiveMS-FCP reduce mean verification rounds by 33.3–46.9% and 38.4–52.2% respectively. ActiveMS-FCP further raises identification success within 16 verification rounds: with MassFormer, from 50.9% to 77.7% on SpectraVerse and from 70.9% to 86.0% on NIST; with ICEBERG, from 87.2% to 93.7% on SpectraVerse and from 88.6% to 95.0% on NIST. These results indicate that feedback-conditioned planning and neural spectrum prediction improve identification along complementary axes: better predictions sharpen the initial ranking, while feedback during verification further reduces the rounds needed to reach the target.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
molecular identification, neural spectrum prediction, active planning, tandem mass spectrometry, AI for science

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_66jpOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The manuscript introduces an active-planning workflow, ActiveMS, for prioritizing candidates from MS2 spectrum prediction for manual verification. A benchmark for active search methods on this task is presented. Two methods for predicting spectra are evaluated in combination with ActiveMS, both resulting in improvements with regard to the necessary rounds of verification.

Critical strengths
  • The application of the active planning idea to spectrum prediction candidates is carried out thoroughly, including three reasonable baselines (besides random and normal ranking also Feedback Greedy), and ablation studies. The benchmark is split into two parts, one based on commercial NIST libraries, the other on a recent, publicly available, and curated spectral library.
Critical weaknesses
  • While improvements on the proposed benchmarks are sound, the practical usefulness of the method is very questionable, hence our recommendation for rejection: It is assumed that we know at which point we have reached identification, which is not the case in practice. Thresholds on spectral similarity or the minimum number of matching peaks are not discussed. Furthermore, the term "identification" should rather be avoided: for the highest level of annotation confidence, we need further orthogonal proof (https://doi.org/10.1021/es5002105); highly similar compounds may result in indistinguishable spectra. The feedback part of the benchmark is based solely on simulated spectra, carrying the risk that the proposed method might not work with "real" reference spectra. The multi-layer neural network architecture of ActiveMS, including cross-attention, could easily lead to overfitting on this particular feedback from simulated spectra. With a large number of high-quality reference spectra publicly available, a smaller complementary benchmark may show that the methods can indeed work also for feedback from non-simulated spectra.
  • Many decisions in the methodology lack motivation or seem rather arbitrary: for instance, three different similarity measures between spectra are used at different points: spectral entropy, cosine, and Euclidean distance, with no explanation. The reason why we need a prioritization method for candidates is not clearly stated, or backed up by references (for example the assumed cost of manual verification, which can only be found in Figure 1, nowhere in the text). The structure-to-spectrum paradigm that spectrum prediction methods are a part of is only one of two main approaches for compound annotation from tandem mass spectra, the other being the spectrum-to-structure paradigm, which is not mentioned or discussed anywhere. The manuscript reads more like another instance of the active planning idea being applied to a new problem. We do not find substantial methodological novelty compared to existing active planning methods.

Minor:

  • y-axis scaling in Figure 3 is very strange and not explained
  • Recheck bibliography: e.g., link to Garnett 2012 paper wrong, same for Jiang 2018
  • "Not Use Feedback" in Figure 1
Questions for authors
  • How does the identification success from the benchmark translate to real-world use cases? What are the criteria to determine success?
  • What are the reasons for the different similarity/distance measures for mass spectra applied throughout the manuscript?
  • What are the methodological novelties of the employed active planning idea compared to existing methods/frameworks?
Reviewer_5MzcOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how feedback from unsuccessful candidate verification can improve MS/MS molecule identification. ActiveMS-GP models feedback residuals over predicted-spectrum similarities, while ActiveMS-FCP learns a feedback-conditioned posterior over candidates. The authors construct benchmarks from SpectraVerse and NIST and evaluate both methods with MassFormer and ICEBERG. Using CFM-ID-generated verification feedback, both methods reduce verification rounds compared with static ranking.

Critical strengths
  1. The paper identifies a useful source of information that static ranking discards. Improvements persist with the stronger ICEBERG predictor, supporting the value of feedback beyond improving the initial ranking.

  2. The geometry ablation and feedback-shuffling experiments support the proposed mechanism. Performance drops when candidate–feedback correspondence is broken, indicating that the trained model uses the verification evidence.

Critical weaknesses
  1. My main concern is the lack of validation with measured feedback. Section 4 computes feedback between CFM-ID predictions for both the candidate and the target, and FCP trains and tests with this same feedback source. Table 10 checks correlation with structural similarity, not agreement with experimental spectral similarity. Although the simulation setting is clearly disclosed, the results leave the central practical question unanswered. Does the reduction in verification rounds persist with real feedback?

  2. FCP’s advantage over GP is less consistent than the main comparison suggests. In Table 8, GP-UCB outperforms FCP on SpectraVerse with ICEBERG in both MVR, 4.18 versus 4.37, and S@R16, 94.75% versus 93.68%. FCP still has better early coverage, so this is a tradeoff rather than uniform superiority. Including the stronger GP configuration in the main comparison would give a clearer assessment of FCP’s added value.

Questions for authors
  1. Could the authors check whether the gains hold with measured feedback on a small subset?

  2. Why was posterior mean chosen for the main GP comparison given UCB’s stronger results?

Reviewer_3g9mOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

 

The authors introduce ActiveMS, an active learning-based framework for choosing target molecules to act as reference samples in tandem mass spectrometry. The authors propose two variants, ActiveMS-GP and ActiveMS-FCP. In a set of experiments, where the authors use CFM-ID 4.0 as a stand-in for experimentally-determined spectra, the authors show that both approaches identify the target molecule faster than baselines including random, static ranking, and a baseline known as feedback greedy. While the paper is interesting and generally well-executed and well-written, there are a couple of improvements I would like to see during the rebuttal phase before voting for acceptance. I will increase my score and champion the paper for acceptance if the following points are addressed during the rebuttal phase: a) the code is released b) the empirical evaluation on the number of rounds required to identify the target can feature multiple random seeds (or a convincing justification for why this is not possible can be given) c) the authors can investigate the incorporation of additional featurizations, such as Morgan fingerprints, fragment count vectors, or SMILES (e.g., via Tanimoto kernel Gaussian processes in the case of the former two representations), can be leveraged to further boost performance.

 

Critical strengths

 

The paper is in general well-executed and targets an interesting real-world domain. If the empirical evaluation is robust, the findings are liable to transfer to real-world scenarios in which feedback is provided by experimentally-determined spectra as opposed to a non-neural simulator.

 

Critical weaknesses

 

I demarcate between major points and minor points below.

 

MAJOR POINTS

 

  1. To aid reproducibility it would be highly desirable if the authors could release their code during the rebuttal phase.

  2. It would be beneficial if the authors could highlight the practical use-case for their workflow compared to tools such as SIRIUS.

  3. It would be worth noting in the manuscript why commonly-used Gaussian process approaches for molecules [1] using e.g., a Tanimoto distance [2] kernel on Morgan fingerprint representations [3] are not used here. It seems as though Morgan fingerprint, SMILES, or graph-based representations could offer improved predictive power and it would be great to see even brief toy experiments to investigate their effect.

  4. For the empirical evaluation are the results reported on a single run? Is it possible to run independent random trials on different data splits and report the standard deviation in the number of rounds required to identify the target?

 

MINOR POINTS

 

  1. There are some missing capitalizations in the references e.g., "Bayesian", "Thompson".

  2. When mentioning Bayesian optimization in Appendix E it would be worth citing the originating works [5, 6] as discussed in [7].

  3. The originating work for Thompson sampling is [8] and should be cited when introducing the method.

  4. The originating work for expected improvement is [9] as discussed in [3] and should be cited when introducing the method.

  5. Additionally, Probability of Improvement (PI) and Upper Confidence Bound (UCB) were first introduced in [5, 6] and should be cited as the originating papers.

 

REFERENCES

 

[1] Griffiths, R.R., Klarner, L., Moss, H., Ravuri, A., Truong, S., Du, Y., Stanton, S., Tom, G., Rankovic, B., Jamasb, A. and Deshwal, A., 2023. GAUCHE: a library for Gaussian processes in chemistry. Advances in Neural Information Processing Systems, 36, pp.76923-76946.

[2] Tanimoto, T.T., 1958. An elementary mathematical theory of classification and prediction.

[3] Rogers, D. and Hahn, M., 2010. Extended-connectivity fingerprints. Journal of chemical information and modeling, 50(5), pp.742-754.

[4] Böcker, S., Letzel, M.C., Lipták, Z. and Pervukhin, A., 2009. SIRIUS: decomposing isotope patterns for metabolite identification. Bioinformatics, 25(2), pp.218-224.

[5] H.J. Kushner (1962). A Versatile Stochastic Model of a Function of Unknown and Time Varying Form. Journal of Mathematical Analysis and Applications 5(1):150–167.

[6] H.J. Kushner (1964). A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise. Journal of Basic Engineering 86(1):97–106.

[7] Garnett, R., Bayesian optimization. Cambridge University Press. 2023.

[8] Thompson, W.R., 1933. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3/4), pp.285-294.

[9] v. r. šaltenis (1971). One Method of Multiextremum Optimization. Avtomatika i Vychislitel’naya Tekhnika (Automatic Control and Computer Sciences) 5(3):33–38.

 

Questions for authors

 

  1. Why are molecular representations such as SMILES, Morgan fingerprints, or fragment count vectors avoided as molecular featurizations here? Would it be possible to include them as featurizations?

 

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.