acceptodds
Under review as a conference paper at ICLR 2027

UncertFP: Uncertainty-Aware Multi-View Evidence Fusion for Molecular Fingerprint Prediction from Tandem Mass Spectra

Abstract

Molecular fingerprint prediction from tandem mass spectrometry (MS/MS) is hindered by incomplete fragmentation and by the fact that distinct molecular structures can produce similar spectra. Existing methods rely mainly on a single view and thus ignore complementary compositional cues. Moreover, similar bit-presence probabilities can reflect different evidence strengths, obscuring whether a prediction has weak or substantial support. Multiple views provide additional structural information, but shared observations can inflate accumulated evidence and conflicting views can undermine prediction. We propose UncertFP, an uncertainty-aware multi-view evidence fusion framework that infers per-bit positive and negative evidence from fragment-peak, neutral-loss, and peak-difference views. Specifically, UncertFP adjusts the contribution of each view through bit reliability and uses a learnable fusion coefficient to control evidence accumulation by interpolating between direct accumulation and weighted averaging. Disagreement discounting further reduces fused evidence when strongly supported views favor opposing bit states. Subsequently, the fused evidence and the shared prior jointly yield a probabilistic fingerprint and an uncertainty mass that quantifies evidence insufficiency. We leverage the relative ranking of whole-fingerprint prediction errors to supervise spectrum-level uncertainty, thereby enabling selective prediction. Across three datasets, two encoders, and three fingerprint dimensionalities, UncertFP improves Soft Tanimoto by 0.07-0.16 over baselines. The complete prediction and scoring pipeline also improves candidate-structure ranking, while uncertainty-based spectrum selection prioritizes predictions with lower average error.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
tandem mass spectrometry, molecular fingerprint prediction, multi-view learning, evidential learning, uncertainty estimation, conflict-aware evidence fusion

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bGREOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes UncertFP, an uncertainty-aware multi-view evidence fusion framework for molecular fingerprint prediction from MS/MS spectra. The authors construct three spectral views and employ an evidence head to estimate bit-wise evidence strength and reliability scores. These are used to determine view-specific fusion weights, while a learnable coefficient controls evidence accumulation and a disagreement discounting mechanism reduces conflicting evidence. The fused evidence is then used to generate fingerprint predictions and uncertainty estimates through Beta distributions. Experiments across three datasets, two encoders, and three fingerprint dimensionalities demonstrate consistent improvements in fingerprint prediction and candidate-structure ranking.

Critical strengths
  1. Distinguishing prediction probability from evidence strength. UncertFP explicitly models bit-presence probability and evidence strength separately, allowing the model to distinguish predictions with similar probabilities but different levels of evidential support. This provides a useful framework for uncertainty estimation and reliability-aware multi-view fusion.

  2. Extensive experimental validation. The authors conduct experiments across three datasets, two encoders, and three fingerprint dimensionalities, demonstrating consistent improvements over the baselines. Comprehensive ablation studies further support the effectiveness of the proposed components. Additionally, candidate-structure ranking and selective prediction experiments demonstrate the effectiveness of uncertainty estimates.

Critical weaknesses
  1. Potentially unfair baseline comparisons. UncertFP and the baselines use different checkpoint selection criteria: the baselines are selected based on validation fingerprint cosine similarity, whereas UncertFP uses validation Soft Tanimoto. Moreover, UncertFP introduces additional trainable parameters through its evidence heads and fusion modules.

  2. Insufficient validation of learned evidence strength and reliability. Both evidence strength and bit-wise reliability scores play central roles in the proposed framework, but the authors do not directly validate whether these learned quantities reflect their intended meanings. In particular, the paper lacks an analysis of whether learned reliability scores correlate with the actual predictive accuracy of individual views. Furthermore, the selective prediction experiments compare uncertainty mass, which receives explicit error-ranking supervision, against predictive entropy, which does not. This makes it difficult to determine whether the improvement comes from evidential modeling or the additional supervision.

Questions for authors
  1. Could the authors provide comparisons with parameter-matched baselines using the same checkpoint selection criteria to clarify whether the improvements stem from the proposed methods rather than additional model capacity or differences in training settings?

  2. Could the authors provide analyses demonstrating that the learned evidence strength and bit-wise reliability scores reflect their intended meanings?

6 comments
AuthorsOpenReview ↗

Response to Reviewer bGRE (1/4)

We sincerely appreciate your valuable feedback.


W1 & Q1: Checkpoint Selection Criteria and Parameter-matched Baselines

R1: To rule out the effects of different checkpoint selection criteria and additional parameters, we conduct experiments on the CANOPUS dataset with 4096-bit fingerprints, using MIST and DiffMS as the baselines, with MIST-4096 and DiffMS-4096 as the baselines. We report Soft Tanimoto and Binary Tanimoto (higher is better ↑\uparrow).

1) Checkpoint selection criteria. We compare the baselines and UncertFP under the same training budget and checkpoint-selection metric.

  • Training budget. UncertFP is initialized from the baseline checkpoints and trained for an additional 300 epochs, so we also train the baselines for an additional 300 training epochs to ensure a fair comparison.
  • Checkpoint-selection metric. The baselines follow the original settings of MIST and DiffMS and use Cosine as the checkpoint-selection metric. For our proposed UncertFP, we use both Cosine and Soft Tanimoto as checkpoint-selection metrics.
ModelAdditional epochsCheckpoint selectionSoft ↑\uparrowBinary ↑\uparrow
MIST-40960Cosine0.340.43
MIST-4096300Cosine0.330.41
UncertFP-MIST-4096300Cosine0.460.49
UncertFP-MIST-4096300Soft Tanimoto0.480.52
ModelAdditional epochsCheckpoint selectionSoft ↑\uparrowBinary ↑\uparrow
DiffMS-40960Cosine0.310.40
DiffMS-4096300Cosine0.290.39
UncertFP-DiffMS-4096300Cosine0.440.46
UncertFP-DiffMS-4096300Soft Tanimoto0.470.51

It can be observed that both MIST-4096 and DiffMS-4096 show a slight decline in performance after an additional 300 epochs of training. One possible explanation is that the extra training causes the models to overfit. By contrast, with the same additional 300 training epochs and Cosine as the checkpoint-selection metric, UncertFP achieves a significant performance improvement over both baselines. This indicates that additional training is not the primary factor for the performance gains, but rather the model design of UncertFP. In addition, UncertFP performs somewhat better when Soft Tanimoto is used instead of Cosine as the checkpoint-selection metric, but the gain is far smaller than that resulting from UncertFP’s novel design. Therefore, the performance improvement stems primarily from our UncertFP, not merely from additional training or a different checkpoint-selection metric.

2) Additional parameters. If the improvements came from increased model capacity, a baseline with the same parameters would perform close to UncertFP. We therefore compare UncertFP with a parameter-matched baseline that uses the same checkpoint-selection metric as UncertFP, and train all models for 300 additional epochs.

  • UncertFP adds an evidence head with about 2.2M parameters and two scalars in the fusion module, compared with the baselines. Its three views share the encoder and the evidence head, so using the neutral-loss (N) and peak-difference (Δ) views in addition to the fragment-peak (F) view adds no parameters.
  • UncertFP-F adds the evidence head of UncertFP to the MIST or DiffMS encoder and, like the baselines, uses only the F view. UncertFP-F differs from UncertFP only in lacking the N and Δ views.
ModelViewsAdditional parametersCheckpoint selectionSoft ↑\uparrowBinary ↑\uparrow
MIST-4096F0Cosine0.330.41
UncertFP-FF2.2MSoft Tanimoto0.360.45
UncertFP-MIST-4096F, N, Δ2.2MSoft Tanimoto0.480.52
ModelViewsAdditional parametersCheckpoint selectionSoft ↑\uparrowBinary ↑\uparrow
DiffMS-4096F0Cosine0.290.39
UncertFP-FF2.2MSoft Tanimoto0.320.41
UncertFP-DiffMS-4096F, N, Δ2.2MSoft Tanimoto0.470.51

It can be observed that UncertFP-F consistently achieves higher scores than the baselines, while UncertFP outperforms UncertFP-F by an even greater margin. This indicates that the gains from increased capacity are smaller than those from the N and Δ views, which do not add parameters. Furthermore, the ablation studies in Section 5.2 and Appendix E of the paper also demonstrate that incorporating the N and Δ views yields greater improvements in both metrics than adding the evidence head.

Therefore, compared with parameter-matched baselines using the same checkpoint selection criteria, the improvements stem primarily from our proposed UncertFP rather than merely from additional model capacity or differences in training settings.

AuthorsOpenReview ↗

Response to Reviewer bGRE (2/4)

W2 & Q2: Intended Meaning of Evidence Strength

R2: Evidence strength is intended to measure how much support each view provides, so it should decrease when the spectra provide less reliable information. To verify this, we conduct experiments on the CANOPUS dataset with 4096-bit fingerprints, using MIST and DiffMS as the baselines, and perturb only the test spectra. Each perturbation is applied at ratios of 20% and 50%, defined as the number of removed, added or corrupted peaks relative to the number of original peaks. We report the Soft Tanimoto of UncertFP (higher is better ↑\uparrow) and Uncertainty, its mean uncertainty mass, which rises as the evidence decreases.

  • None denotes the unperturbed test spectra.
  • Drop peaks removes randomly selected peaks to simulate missing fragments.
  • Add noise peaks inserts low-intensity peaks annotated with random subformulae of the precursor formula to simulate noisy peaks.
  • Corrupt subformulae replaces the subformula annotations of selected peaks with other subformulae of the precursor formula to simulate incorrect subformula annotations.
UncertFP-MIST-4096UncertFP-DiffMS-4096
PerturbationRatioSoft ↑\uparrowUncertaintySoft ↑\uparrowUncertainty
None-0.480.120.470.15
Drop peaks20%0.450.150.430.19
50%0.390.230.380.25
Add noise peaks20%0.460.140.450.16
50%0.440.170.440.18
Corrupt subformulae20%0.430.180.420.22
50%0.360.290.350.28

It can be observed that, the mean uncertainty mass rises with the perturbation ratio and is higher in the settings with larger drops in Soft Tanimoto. The learned evidence therefore decreases when the spectra provide less reliable information, in line with the intended meaning of evidence strength.

AuthorsOpenReview ↗

Response to Reviewer bGRE (3/4)

W2 & Q2 (cont.): Intended Meaning of Bit-wise Reliability Scores

R3: Reliability scores are intended to indicate how reliable each view is on each fingerprint bit, so a view with a higher reliability score on a bit should predict that bit more accurately. To verify this, we test whether the learned reliability scores correlate with the actual predictive accuracy of individual views. Reliability diagrams [1] check whether a confidence score reflects accuracy by grouping predictions by the score and comparing the accuracy across groups. Following this practice, we group the view predictions of UncertFP by their reliability scores and compare the precision across groups. We conduct the analysis on the test set of the CANOPUS dataset with 4096-bit fingerprints, using MIST and DiffMS as the baselines.

The details of the experimental setup above are explained as follows:

  • Reliability scores. For each spectrum in the test set, the evidence head of UncertFP maps the representation of each of the three views to one reliability score per fingerprint bit.
  • Selected predictions. To compare only predictions with high and similar bit-presence probabilities, we take every view prediction with a bit-presence probability of at least 0.9, in which a view confidently predicts the bit to be present.
  • Groups. A prediction is labeled Highest if the reliability score of its view is the largest of the three views on that bit and Lowest if it is the smallest. To check whether the reliability scores encode a fixed preference for one view, we form these groups over all views and within each of the fragment-peak (F), neutral-loss (N) and peak-difference (Δ) views.
  • Precision. The precision of a group is the fraction of its predictions whose bit is present in the true fingerprint (higher is better ↑\uparrow).
UncertFP-MIST-4096UncertFP-DiffMS-4096
ViewRank of reliability scorePrecision ↑\uparrowPrecision ↑\uparrow
All viewsLowest0.720.70
Highest0.840.85
FLowest0.740.72
Highest0.860.86
NLowest0.710.70
Highest0.840.85
ΔLowest0.700.69
Highest0.820.83

It can be observed that the mean bit-presence probabilities of the groups differ by at most 0.02, so the differences in precision between the groups do not come from differences in bit-presence probability.

Furthermore, over all views, the predictions from the view with the highest reliability score on a bit have higher precision than those from the view with the lowest reliability score for both baselines. Since the bitwise reliability weights increase with the reliability score, the fused prediction favors the view that is more accurate on that bit.

Additionally, within each of the F, N and Δ views, the predictions with the highest reliability score also have higher precision than those with the lowest. This indicates that the reliability scores follow the accuracy of a view on each bit rather than encoding a fixed preference for one view.

Therefore, the learned reliability scores correlate with the actual predictive accuracy of individual views, which reflects their intended meaning.

Reviewer_TnD6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes UncertFP for molecular fingerprint prediction from MS/MS spectra. It combines fragment-peak, neutral-loss, and peak-difference views through bitwise evidence weighting, disagreement discounting, and Beta-distribution-based uncertainty estimation.

Critical strengths
  1. The paper addresses a meaningful problem by explicitly modeling evidence strength and disagreement across spectral representations.
  2. The experiments are extensive, covering three datasets, two spectral encoders, and three fingerprint dimensionalities, with consistent improvements in fingerprint prediction.
  3. The appendix provides substantial ablations of spectral views, fusion strategies, training objectives, and uncertainty mechanisms, supporting the effectiveness of the complete framework.
Critical weaknesses
  1. The main comparisons do not fully control for training procedures and optimization budgets. According to Appendix C.4, UncertFP initializes from pretrained baseline checkpoints and undergoes up to 300 additional training epochs. Furthermore, baseline checkpoints are selected using fingerprint cosine similarity, whereas UncertFP uses validation Soft Tanimoto. Consequently, the improvements in Table 1 may partially reflect additional optimization and different checkpoint-selection criteria. Although the internal fusion ablations are useful, a directly matched baseline with the same training budget, losses, and selection metric is needed to establish the full performance gain.

  2. The benefit of explicitly separating spectral information into three views is not sufficiently distinguished from increased model capacity. All three views are deterministic transformations of the same input spectrum, and existing formula-aware models such as MIST already incorporate inter-peak relationships. Although Tables 2 and the supplementary ablations show complementary predictive behavior, they do not establish that separate evidential representations are preferable to a single model jointly encoding the same information. A capacity-matched single-encoder alternative with access to all three feature types would help isolate the contribution of the proposed multi-view formulation.

  3. The interpretation of uncertainty mass is not sufficiently validated. The paper interprets Beta evidence strength as uncertainty due to insufficient information. However, the training objective explicitly supervises spectrum-level uncertainty through the ranking of prediction errors, making it unclear whether the resulting measure captures meaningful evidential uncertainty or simply learns an error-prediction score. The uncertainty evaluation focuses on selective risk, primarily on CANOPUS, without systematic calibration analysis or comparisons against stronger confidence estimators such as ensemble uncertainty. Therefore, the probabilistic interpretation of the learned evidence remains insufficiently established.

  4. The contribution of uncertainty-aware candidate scoring is not isolated from improved fingerprint prediction. The complete retrieval pipeline combines improved fingerprint probabilities, uncertainty-weighted Tanimoto similarity, and a Beta-sampling-based variance penalty. Although Appendix D reports consistent retrieval improvements, it does not separately evaluate these scoring components while holding the predicted fingerprints fixed. Therefore, it is unclear how much of the downstream retrieval gain comes from better fingerprint predictions versus the proposed uncertainty-aware ranking mechanism.

Questions for authors

See weakness.

7 comments
AuthorsOpenReview ↗

Response to Reviewer TnD6 (1/5)

We sincerely appreciate your valuable feedback.


W1: Training Budget, Losses and Selection Metric

R1: To rule out the effects of additional training, different losses and checkpoint-selection metric, we conduct experiments on the CANOPUS dataset with 4096-bit fingerprints, using MIST and DiffMS as baselines (i.e., MIST-4096 and DiffMS-4096). We report Soft Tanimoto and Binary Tanimoto (higher is better ↑\uparrow).

  • Training budget. UncertFP is initialized from the baseline checkpoints and trained for an additional 300 epochs, so we also train the baselines for an additional 300 training epochs to ensure a fair comparison.

  • Losses. In these additional epochs, the baselines are trained with the fingerprint losses of UncertFP, namely the class-weighted BCE loss LBCE\mathcal{L}_{\mathrm{BCE}} and the Soft Tanimoto loss Lsim\mathcal{L} _{\mathrm{sim}}. The ranking loss Lrank\mathcal{L} _{\mathrm{rank}} of UncertFP supervises its uncertainty mass, which the baselines do not output, so we report UncertFP trained with and without it.

  • Checkpoint-selection metric. Because the original MIST and DiffMS baselines both use the Cosine checkpoint-selection metric, we follow their original settings for the baselines. For our proposed UncertFP, we use both Cosine and Soft Tanimoto as checkpoint-selection metrics.

ModelAdditional epochsWith Ranking loss Lrank\mathcal{L}_{\mathrm{rank}}Checkpoint selectionSoft ↑\uparrowBinary ↑\uparrow
MIST-40960NoCosine0.340.43
MIST-4096300NoCosine0.330.41
UncertFP-MIST-4096300NoCosine0.440.47
UncertFP-MIST-4096300YesCosine0.460.49
UncertFP-MIST-4096300YesSoft Tanimoto0.480.52
ModelAdditional epochsWith Ranking loss Lrank\mathcal{L}_{\mathrm{rank}}Checkpoint selectionSoft ↑\uparrowBinary ↑\uparrow
DiffMS-40960NoCosine0.310.40
DiffMS-4096300NoCosine0.290.39
UncertFP-DiffMS-4096300NoCosine0.410.43
UncertFP-DiffMS-4096300YesCosine0.440.46
UncertFP-DiffMS-4096300YesSoft Tanimoto0.470.51

It can be observed that both MIST-4096 and DiffMS-4096 show a slight decline in performance after an additional 300 epochs of training. One possible explanation is that the extra training causes the models to overfit.
By contrast, with the same additional 300 training epochs, the same losses and Cosine as the checkpoint-selection metric, UncertFP trained without the ranking loss Lrank\mathcal{L}_{\mathrm{rank}} achieves a significant performance improvement over both baselines. This indicates that additional training and different losses are not the primary factors for the performance gains, but rather the model design of UncertFP. Adding the ranking loss raises both metrics further.

In addition, UncertFP performs better when Soft Tanimoto is used instead of Cosine as the checkpoint-selection metric, but the gain is far smaller than that resulting from UncertFP’s novel design.

Therefore, the performance improvement stems primarily from our UncertFP, not merely from additional training, different losses or a different checkpoint-selection metric.

AuthorsOpenReview ↗

Response to Reviewer TnD6 (2/5)

W2: Separate Views or Increased Model Capacity

R2: To distinguish the benefit of separating spectral information into three views from increased model capacity, we conduct experiments on the CANOPUS dataset with 4096-bit fingerprints, using MIST and DiffMS as the baselines (i.e., MIST-4096 and DiffMS-4096). If the improvements came from increased model capacity, a baseline with the same parameters would perform close to UncertFP. We therefore compare UncertFP with two capacity-matched baselines that use the same checkpoint selection criterion as UncertFP, and train all models for 300 additional epochs. We report Soft Tanimoto and Binary Tanimoto (higher is better ↑\uparrow).

  • UncertFP adds an evidence head with about 2.2M parameters and two scalars in the fusion module, compared with the baselines. Its three views share the encoder and the evidence head, so using the neutral-loss (N) and peak-difference (Δ) views in addition to the fragment-peak (F) view adds no parameters.
  • UncertFP-F adds the evidence head of UncertFP to the baseline encoder and, like the baselines, uses only the F view. UncertFP-F differs from UncertFP only in lacking the N and Δ views.
  • Self-attention, which we construct, is the single-encoder alternative with access to all three feature types. It concatenates the tokens of the F, N and Δ views into one sequence, fuses them in a single pass through the encoder, and feeds the result to a single evidence head. Self-attention differs from UncertFP only in how the three views are fused.
ModelViewsAdditional parametersCheckpoint selectionSoft ↑\uparrowBinary ↑\uparrow
MIST-4096F0Cosine0.330.41
UncertFP-FF2.2MSoft Tanimoto0.360.45
Self-attentionF, N, Δ2.2MSoft Tanimoto0.410.48
UncertFP-MIST-4096F, N, Δ2.2MSoft Tanimoto0.480.52
ModelViewsAdditional parametersCheckpoint selectionSoft ↑\uparrowBinary ↑\uparrow
DiffMS-4096F0Cosine0.290.39
UncertFP-FF2.2MSoft Tanimoto0.320.41
Self-attentionF, N, Δ2.2MSoft Tanimoto0.370.45
UncertFP-DiffMS-4096F, N, Δ2.2MSoft Tanimoto0.470.51

It can be observed that UncertFP-F consistently achieves higher scores than the baselines, while UncertFP outperforms UncertFP-F by an even greater margin. This indicates that the gains from increased capacity are smaller than those from the N and Δ views, which do not add parameters. Furthermore, the ablation studies in Section 5.2 and Appendix E of the paper also demonstrate that incorporating the N and Δ views yields greater improvements in both metrics than adding the evidential head.

Although the three views are deterministic transformations of the same spectrum and MIST already incorporates inter-peak relationships, adding the N and Δ views to UncertFP still significantly improves both metrics compared to MIST-4096. The reason is that UncertFP constructs these views in a way that provides positive and negative evidence to each fingerprint bit from fragment, precursor–fragment and interfragment information, respectively.

Even with the same number of parameters and the same three views, Self-attention scores higher than UncertFP-F on both metrics, but consistently lower than UncertFP. Unlike Self-attention, UncertFP keeps a separate evidential representation for each view and fuses them with bitwise reliability weights and disagreement discounting.

Therefore, separating spectral information into three views brings a benefit beyond increased model capacity, and separate evidential representations are preferable to a single model jointly encoding the same information. This capacity-matched comparison isolates the contribution of the proposed multi-view formulation.

AuthorsOpenReview ↗

Response to Reviewer TnD6 (3/5)

W3: Interpretation of Uncertainty Mass

R3: To test whether the uncertainty mass captures meaningful evidential uncertainty or simply learns an error-prediction score, we compare it with baseline ranking scores, including an error-prediction score and ensemble uncertainty. We conduct the experiments on the CANOPUS, MassSpecGym and SpectraVerse-positive datasets with 4096-bit fingerprints, using MIST and DiffMS as the baselines. Each ranking score keeps the 50% of test spectra that it ranks as most reliable, and the selective risk is the mean 1−Soft Tanimoto of the kept spectra (lower is better ↓\downarrow).

UncertFP-BS is UncertFP trained without the ranking loss Lrank\mathcal{L}_{\mathrm{rank}}, so its uncertainty receives no error supervision. Deep ensemble [1] consists of five MIST-4096 or DiffMS-4096 models trained with different random seeds and averages their bit probabilities. Uncertainty denotes the mean uncertainty mass.

The ranking scores are as follows:

  • Random is a random ranking whose expected risk equals the mean error of the model.
  • Entropy is the mean predictive entropy.
  • Mutual information is the entropy of the averaged probabilities minus the mean entropy of the five members, which measures their disagreement.
  • MLP is a supervised error-prediction model that follows ConfidNet [2] and is trained on frozen UncertFP features to predict the error of each spectrum.

The p-values in parentheses come from a permutation test against 2000 random rankings. In each column, the lowest risk is in bold, and the lowest risk among the ranking scores without error supervision is in italics.

MIST-4096DiffMS-4096
ModelRanking scoreWith Error supervisionCANOPUS ↓\downarrowMassSpecGym ↓\downarrowSpectraVerse-positive ↓\downarrowCANOPUS ↓\downarrowMassSpecGym ↓\downarrowSpectraVerse-positive ↓\downarrow
Deep ensembleRandom/0.660.760.730.690.650.71
EntropyNo0.610.740.700.640.630.67
Mutual informationNo0.630.750.710.660.640.69
UncertFP-BSRandom/0.530.630.620.540.590.57
EntropyNo0.480.600.570.490.560.53
UncertaintyNo0.45 (p<0.001)0.57 (p<0.001)0.55 (p<0.001)0.46 (p<0.001)0.53 (p<0.001)0.51 (p<0.001)
UncertFPRandom/0.520.620.610.530.580.56
MLPYes0.440.550.540.440.520.50
UncertaintyYes0.430.520.510.440.480.47

It can be observed that without error supervision, i.e., when the ranking loss Lrank\mathcal{L}_{\mathrm{rank}} is removed, the Uncertainty of UncertFP-BS has lower risk than Entropy and Random (p<0.001). While under error supervision, in five out of the six columns, the risk of UncertFP using an MLP as the ranking score is higher than that of UncertFP using Uncertainty.

Additionally, regarding the comparison between Deep Ensembles and UncertFP, since they have different mean errors, we compare the risk reduction of each ranking score relative to Random of the same model. Specifically, UncertFP-BS with the Uncertainty reduces the risk more than the best ensemble score, Entropy, even without error supervision, and UncertFP with the Uncertainty reduces it further.

Therefore, the uncertainty mass captures meaningful evidential uncertainty rather than simply learning an error-prediction score, and it ranks prediction errors better than ensemble uncertainty.

Reviewer_oZfYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper introduces UncertFP, a multi-view framework for molecular fingerprint prediction from MS/MS spectra. It combines fragment peaks, neutral losses, and peak differences through reliability weighting, evidence aggregation, and disagreement discounting. A Beta-based uncertainty formulation supports fingerprint prediction and spectrum selection. Experiments across three datasets and two encoders report improved Tanimoto scores, candidate ranking, and selective prediction.

Critical strengths
  • The paper combines complementary spectral representations to improve molecular fingerprint prediction.
  • The authors report consistent improvements across three datasets, two encoders, and multiple fingerprint sizes. The extensive ablations also help assess individual components.
  • The finding connects fingerprint prediction with uncertainty-based spectrum selection.
Critical weaknesses
  • UncertFP receives additional training and uses a different checkpoint-selection metric. It is unclear whether the improvements come from the proposed method or these advantages.
  • Reliability weighting, evidence aggregation, and conflict discounting largely build on existing methods. This lacks comparisons with strong attention-based or established fusion methods.
  • The model treats learned neural outputs as evidence without validating their statistical significance.
  • The ranking loss directly trains uncertainty to follow prediction errors, yet the evaluation compares it with unsupervised entropy. This does not establish the benefit of evidential uncertainty.
Questions for authors
  • How does the model handle cases where all three views confidently agree on an incorrect prediction?
  • How robust is the framework to noisy peaks, missing fragments, and incorrect subformula annotations?
  • How does performance shift when the model uses a global fusion coefficient despite varying dependencies between views and fingerprint bits?
  • How does the proposed uncertainty measure compare with a supervised error-prediction model?
  • What are the results of uncertainty-based selection on other datasets than CANOPUS?
7 comments
AuthorsOpenReview ↗

Response to Reviewer oZfY (1/4)

We sincerely appreciate your valuable feedback.


W1: Additional Training and Checkpoint-Selection Metric

R1: To rule out the effects of additional training and checkpoint-selection metric, we conduct experiments on the CANOPUS dataset with 4,096-bit fingerprints, using MIST and DiffMS as baselines (i.e., MIST-4096 and DiffMS-4096). Because the original MIST and DiffMS baselines both use the Cosine checkpoint-selection metric, we follow their original settings and train the baselines for an additional 300 training epochs to ensure a fair comparison. For our proposed UncertFP, it are trained with additional 300 epochs, using both Cosine and Soft Tanimoto as checkpoint-selection metrics. We report the Soft Tanimoto and Binary Tanimoto evaluation metrics, for which higher values are better. The results are shown in the table below:

MethodAdditional epochsCheckpoint selectionSoft ↑\uparrowBinary ↑\uparrow
MIST-40960Cosine0.340.43
MIST-4096300Cosine0.330.41
UncertFP-MIST-4096300Cosine0.460.49
UncertFP-MIST-4096300Soft Tanimoto0.480.52
MethodAdditional epochsCheckpoint selectionSoft ↑\uparrowBinary ↑\uparrow
DiffMS-40960Cosine0.310.40
DiffMS-4096300Cosine0.290.39
UncertFP-DiffMS-4096300Cosine0.440.46
UncertFP-DiffMS-4096300Soft Tanimoto0.470.51

It can be observed that both MIST-4096 and DiffMS-4096 show a slight decline in performance after an additional 300 epochs of training. One possible explanation is that the extra training causes the models to overfit. By contrast, with the same additional 300 training epochs and Cosine as the checkpoint-selection metric, UncertFP achieves a significant performance improvement over both baselines. This indicates that additional training is not the primary factor for the performance gains, but rather the model design of UncertFP. In addition, UncertFP performs somewhat better when Soft Tanimoto is used instead of Cosine as the checkpoint-selection metric, but the gain is far smaller than that resulting from the UncertFP’s novel design.

Therefore, the performance improvement stems primarily from our UncertFP, not merely from additional training or a different checkpoint-selection metric.


W2: Comparisons with Attention-based or Established Fusion Methods

R2: We conduct experiments on the CANOPUS dataset with 4096-bit fingerprints, using MIST and DiffMS as the baselines. We replace the fusion strategy of UncertFP with Self-attention, an attention-based method that we construct, and with those of the established fusion methods TMC [1], ECML [2] and DBF [3] (Already cited in the main text). All variants differ from UncertFP only in the fusion strategy. We report Soft Tanimoto and Binary Tanimoto (higher is better ↑\uparrow).

  • None denotes the baselines MIST and DiffMS, which use only the fragment-peak view without fusion.
  • Self-attention concatenates the tokens of the fragment-peak, neutral-loss and peak-difference views into one sequence, fuses them in a single pass through the encoder, and feeds the result to a single evidence head.
  • TMC combines the opinions of the views with Dempster's combination rule.
  • ECML aggregates the opinions through an operation equivalent to evidence averaging.
  • DBF measures conflict using probability distances and joint certainty, discounts individual opinions, and then applies generalized averaging.
MIST-4096DiffMS-4096
Fusion strategySoft ↑\uparrowBinary ↑\uparrowSoft ↑\uparrowBinary ↑\uparrow
None0.340.430.310.40
Self-attention0.410.480.370.45
TMC0.420.490.380.46
ECML0.400.480.360.43
DBF0.440.500.450.49
UncertFP0.480.520.470.51

It can be observed that UncertFP consistently achieves the best performance. Unlike the fusion methods mentioned above, UncertFP assigns view contributions with bitwise reliability weights and controls the accumulation of shared evidence with the learnable fusion coefficient η\eta. It then discounts the aggregated evidence according to the disagreement at each bit. The ablation experiments in Section H of the appendix also show that replacing the learned η\eta with a fixed value or removing the disagreement discount reduces both metrics. With these designs, UncertFP achieves the best results among the compared fusion strategies, which demonstrates the effectiveness of the proposed fusion strategy and its advantage over attention-based and established fusion methods.

AuthorsOpenReview ↗

Response to Reviewer oZfY (2/4)

W3: Statistical Significance of Learned Evidence

R3: As you note, UncertFP treats learned neural outputs as evidence, as do evidential deep learning [4] and the multi-view methods TMC [1], ECML [2] and DBF [3]. Whether such learned evidence is faithful remains an open research question [5, 6]. We validate the learned evidence from two aspects.

1) Error ranking without error supervision. In R4, we remove the ranking loss from training and rank the test spectra by the mean uncertainty mass of the resulting model, UncertFP-BS. For both baselines on all three datasets, its selective risk is lower than that of all 2000 random rankings (permutation test, p<0.001). The learned evidence therefore carries statistically significant information about prediction errors without being trained on them.

2) Response to degraded spectra. In R6, we drop peaks, add noise peaks or corrupt subformula annotations in the test spectra. The mean uncertainty mass rises with the perturbation ratio and is higher in the settings with larger drops in Soft Tanimoto. The learned evidence therefore decreases when the spectra provide less reliable information.


W4 & Q4: Benefit of Evidential Uncertainty

R4: To establish the benefit of evidential uncertainty, we compare it with baseline ranking scores under matched error supervision. We conduct the experiments on the CANOPUS, MassSpecGym and SpectraVerse-positive datasets with 4096-bit fingerprints, using MIST and DiffMS as the baselines. Each ranking score keeps the 50% of test spectra that it ranks as most reliable, and the selective risk is the mean 1−Soft Tanimoto of the kept spectra (lower is better ↓\downarrow).

UncertFP-BS is UncertFP trained without the ranking loss, so its uncertainty receives no error supervision.

Uncertainty denotes the mean uncertainty mass. The baseline ranking scores are Random, a random ranking whose expected risk equals the mean error of the model; Entropy, the mean predictive entropy; and MLP, a supervised error-prediction model that follows ConfidNet [7] and is trained on frozen UncertFP features to predict the error of each spectrum. The p-values in parentheses come from a permutation test against 2000 random rankings.

MIST-4096DiffMS-4096
ModelRanking scoreError supervisionCANOPUSMassSpecGymSpectraVerse-positiveCANOPUSMassSpecGymSpectraVerse-positive
UncertFP-BSRandom/0.530.630.620.540.590.57
EntropyNo0.480.600.570.490.560.53
UncertaintyNo0.45 (p<0.001)0.57 (p<0.001)0.55 (p<0.001)0.46 (p<0.001)0.53 (p<0.001)0.51 (p<0.001)
UncertFPRandom/0.520.620.610.530.580.56
EntropyNo0.490.570.560.490.540.51
MLPYes0.440.550.540.440.520.50
UncertaintyYes0.430.520.510.440.480.47

It can be observed that, without error supervision, the Uncertainty of UncertFP-BS has lower risk than Entropy and Random for both baselines on all three datasets (p<0.001), so evidential uncertainty outperforms entropy. With error supervision, MLP has higher risk than the Uncertainty of UncertFP in five of the six columns and matches it for DiffMS-4096 on the CANOPUS dataset, so evidential uncertainty also outperforms a supervised error-prediction model. These comparisons under matched supervision establish the benefit of evidential uncertainty.


Q1: Views Confidently Agreeing on an Incorrect Prediction

R5: UncertFP cannot identify an incorrect prediction on which all three views confidently agree and provide abundant evidence. Such errors produce no disagreement between views, so every method that estimates reliability from the agreement between views faces this problem. However, confident agreement does not imply abundant evidence, because UncertFP estimates the evidence strength of each view separately from its bit-presence probability. The uncertainty mass rises as the total evidence decreases. Therefore, when the three views confidently agree on an incorrect prediction but provide little evidence, the uncertainty mass remains high and marks the prediction as unreliable.

AuthorsOpenReview ↗

Response to Reviewer oZfY (3/4)

Q2: Robustness to Noisy Peaks, Missing Fragments and Incorrect Annotations

R6: We conduct experiments on the CANOPUS dataset with 4096-bit fingerprints, using MIST and DiffMS as the baselines. We perturb only the test spectra and evaluate the baselines and UncertFP on the same perturbed spectra. Each perturbation is applied at ratios of 20% and 50%, defined as the number of removed, added or corrupted peaks relative to the number of original peaks. We report Soft Tanimoto (higher is better ↑\uparrow), together with the mean uncertainty mass of UncertFP in parentheses.

  • None denotes the unperturbed test spectra.
  • Drop peaks removes randomly selected peaks to simulate missing fragments.
  • Add noise peaks inserts low-intensity peaks annotated with random subformulae of the precursor formula to simulate noisy peaks.
  • Corrupt subformulae replaces the subformula annotations of selected peaks with other subformulae of the precursor formula to simulate incorrect subformula annotations.
PerturbationRatioMIST-4096UncertFP-MIST-4096 (Uncertainty)DiffMS-4096UncertFP-DiffMS-4096 (Uncertainty)
None-0.340.48 (0.12)0.310.47 (0.15)
Drop peaks20%0.320.45 (0.15)0.280.43 (0.19)
50%0.290.39 (0.23)0.250.38 (0.25)
Add noise peaks20%0.330.46 (0.14)0.300.45 (0.16)
50%0.310.44 (0.17)0.290.44 (0.18)
Corrupt subformulae20%0.300.43 (0.18)0.270.42 (0.22)
50%0.240.36 (0.29)0.220.35 (0.28)

It can be observed that, for all three perturbations, the Soft Tanimoto of both the baselines and UncertFP decreases as the perturbation ratio increases, while UncertFP remains above the corresponding baseline in every perturbed setting. For both baselines, the mean uncertainty mass of UncertFP rises with the perturbation ratio and is higher in the settings with larger drops in Soft Tanimoto. Therefore, UncertFP keeps its advantage over the baselines under noisy peaks, missing fragments and incorrect subformula annotations, and its uncertainty reflects the reduced reliability of its predictions. This demonstrates the robustness of our UncertFP to noisy peaks, missing fragments, and incorrect subformula annotations.


Q3: Global Fusion Coefficient despite Varying Dependencies

R7: UncertFP fuses the evidence of the three views with a global learnable fusion coefficient η\eta, which interpolates between direct accumulation and bitwise reliability-weighted averaging. We use a global coefficient because the varying dependencies between views and fingerprint bits are modeled by the bitwise reliability weights λv,k\lambda_{v,k} (in Eq.3). Within the weighted average, λv,k\lambda_{v,k} controls the contribution of each view to each bit, while η\eta controls the total evidence and the influence of the learned weights.

To verify this design, we conduct experiments on the CANOPUS dataset with 4096-bit fingerprints, using MIST and DiffMS as the baselines. We replace the global η\eta with one learnable coefficient ηk\eta_k per bit and keep all other settings unchanged. We report Soft Tanimoto and Binary Tanimoto (higher is better ↑\uparrow).

MIST-4096DiffMS-4096
UncertFPSoft ↑\uparrowBinary ↑\uparrowSoft ↑\uparrowBinary ↑\uparrow
Global η\eta0.480.520.470.51
ηk\eta_k per bit0.480.510.470.50

It can be observed that per-bit coefficients keep Soft Tanimoto unchanged while only slightly reducing Binary Tanimoto for both baselines. Therefore, despite the varying dependencies between views and fingerprint bits, a global fusion coefficient does not reduce performance, which verifies this design.


Q5: Uncertainty-based Selection on Other Datasets

R8: We evaluate uncertainty-based selection on the MassSpecGym and SpectraVerse-positive datasets with 1024-, 2048- and 4096-bit fingerprints, using MIST and DiffMS as the baselines. On MassSpecGym, we also evaluate the released 4096-bit weights of both baselines, giving 8 and 6 configurations on the two datasets, respectively. The results are shown in the newly added Figures 5 and 6 of Appendix C.7.

Uncertainty-mass ranking retains the spectra with the lowest mean uncertainty mass, and entropy ranking retains those with the lowest mean predictive entropy. The figures plot selective risk (lower is better) against coverage, the fraction of test spectra retained, and the shaded band marks the 95% range of 2000 random rankings.

In all 14 configurations, uncertainty-mass ranking has lower selective risk than entropy ranking and stays below the 95% range of random ranking at both 25% and 50% coverage. The revised Section 5.3 reports these results together with those on the CANOPUS dataset. Therefore, the benefit of uncertainty-based selection is not limited to the CANOPUS dataset, but across all datasets.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.