acceptodds
Under review as a conference paper at ICLR 2027

Iona: Scaling Law for Mass Spectrometry Foundation Model

Abstract

Mass spectrometry is a core analytical technique for identifying chemical molecules, with applications ranging from cancer biomarker and metabolite detection to heavy isotope analysis, pathogen identification, and explosive screening at airports. Building machine learning models that genuinely understand mass spectra is therefore an important problem. Recent transformer models trained on tandem mass spectra span roughly 1M to 400M parameters, yet a basic question remains unanswered: *do larger models actually understand spectra better, and how large is large enough?* No published spectrum model reports a systematic size sweep or a fitted scaling law. In this work, we conduct the **first such study for self-supervised mass spectrometry foundation models**. We present **Iona**, a family of encoders pretrained at 25M, 50M, 100M, 200M, and 400M parameters on MSConsensus-100M, a corpus of 100 million consensus MS/MS spectra, using a masked intensity-reconstruction objective. Evaluation loss follows a power law in pretraining compute, with a compute-efficient frontier on which each larger model overtakes the previous one, and extrapolating the frontier places the compute-optimal model size well beyond the largest spectrum transformer published to date; we therefore aim to push Iona past 1 billion parameters next. Capability grows with scale downstream as well, from denoising and retrieval to chimeric-spectrum decomposition, and with simple fine-tuning Iona reaches or surpasses strong specialised models in real identification pipelines: it beats a supervised proteomics encoder on real chimeric spectra, adds 20,763 peptide–spectrum matches at 1% FDR to database search, and recovers 37–60% more peptides from the weakest single-cell acquisitions. Random-initialisation controls show that this performance comes from pretraining, not fine-tuning. We further test a previously unexamined hypothesis: *can a model succeed without encoding any absolute ?* In our design, all mass information enters through a learned Fourier-feature bias on pairwise mass differences, making the encoder translation-invariant in . Contrary to the assumption that absolute mass positioning is necessary, this relative-only design solves every task above, and its learned bias recovers amino-acid residue, neutral-loss and isotope masses without supervision: the model learns chemical reality from data. Because the recipe uses no peptide labels, annotations or any other proteomics-specific ingredient, we expect the same scaling behaviour to hold for metabolomics and other mass spectrometry applications.

Primary area:
foundation or frontier models, including LLMs
Keywords:
foundational model, mass spectrometry, proteomics, scaling law

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GBnLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose IONA, a group of self-supervised models that offer a systematic scaling analysis for mass spectrometry data. Their contributions are twofold:

  1. They propose a scaling law for the number of parameters of the foundation models (one for each set of parameters) in respect to loss.
  2. Instead of focusing the model's attention on the mass of the molecule itself that other self supervised models do for MS data, they are focusing the attention of the model to the differences between peak masses. So the model has no information about the absolute mass, just normalized intensities and pair wise mass differences between peaks.

The models are masked transformers that learn by combining visible intensities with mass relationships across the whole spectrum to predict the masked intensities.

They perform various experiments across tasks and scales showing the limits and capabilities of the proposed models.

Critical strengths
  1. Interpretability is embedded in the architecture by all the information flowing though a single bias function per attention head. So the model's learned chemistry can be interpreted directly. They show its significance with an experiment (including a negative control) in section 4.2.
  2. The authors perform a great number of experiments and the proposed methodology is evaluated across multiple data, tasks and hypotheses testing, creating a very well rounded picture of the models' capabilities.
  3. The authors fit a two-term equation (scaling law) to all 40 checkpoints with near-perfect accuracy. This led to showing that data drives improvement roughly four times faster than model size, and they use it to make a concrete, testable prediction about compute-optimal model size that they then check independently on held-out runs. After this analysis they acknowledge the limitation of the their size as suboptimal.
Critical weaknesses

The authors don't compared their method of including only mass differences and not absolute masses with other methodologies that they include absolute masses. They include this limitation in the paper, but since this is the main novelty of the proposed design an explanation of what they expect to see with this comparison or a small experiment /ablation including absolute masses would strengthen the paper. It is unclear whether the invariance to common mass shift helps transfer or limits performance, particularly on tasks where absolute positions are diagnostic.

Questions for authors
  1. For the classification task I wanted to ask the authors to perform an additional experiment, removing the pretraining from the model and performing the task in a supervised manner to compare it against IONA and Casanovo.
  2. See weakness 2: a small ablation or explanation on comparison with models that have attention on absolute masses.
1 comment

We thank the reviewer for the positive assessment and constructive suggestions.

“The authors don’t compare their method of including only mass differences and not absolute masses [...]”

We have added a matched 25M ablation using absolute m/z only (Section 4.3, Appendix S). The absolute variant fits pretraining slightly better (0.0658 vs. 0.0688, single seed), but this gap is small, less than half the improvement from doubling model size (0.0073 from 25M to 50M). We therefore do not claim superiority on clean, well-calibrated data. On whether invariance helps transfer, we evaluated both encoders on denoising under test-time drift (Table 25). The relative-only encoder is unaffected by constant offsets, where the absolute variant loses up to 15% AUROC. Under ppm drift, it degrades 5–9× less.

For tasks where absolute positions are diagnostic, we keep the encoder shift-invariant and provide absolute information downstream when a task needs it. In our new de novo experiment, for example, only the decoder receives the precursor mass, and the relative-only encoder still exceeds Casanovo v5 in mean peptide recall (Section 4.4, Appendix T). We leave a combined absolute + relative encoder to future work.

“[...] removing the pretraining from the model and performing the task in a supervised manner to compare it against IONA and Casanovo.”

We thank the reviewer for this suggestion. We ran this control on a new, larger chimericity benchmark with 101 PRIDE projects split by project. We also added a head with no encoder to separate the encoder’s contribution (Section 4.4, Table 28).

Trained purely supervised, the Iona architecture is no better than the encoder-free head (mean project AUROC 0.727 vs. 0.733; Table 28). Pretrained Iona-25M reaches 0.832, and Iona-400M reaches 0.872, against 0.834 for fine-tuned CasanovoTL. The same pattern holds for DIA precursor prediction, with bin AP of 0.533 from random initialization vs. 0.698 for pretrained Iona-25M (Table 29).

Reviewer_8g4qOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose Iona, a family of MS/MS foundation models pretrained using masked-intensity reconstruction, and fit a scaling law relating model size, training compute, and reconstruction loss. They evaluate the learned representations on several downstream tasks, including denoising, spectrum retrieval, chimera decomposition, PSM rescoring, and fragment-intensity prediction. They also investigate an architecture encoding only pairwise m/z differences, without absolute mass information.

Critical strengths

Establishing how pretraining scale affects downstream performance could be valuable for this relatively understudied modality, and some applications shown are interesting.

Repurposing a pretrained spectrum encoder for peptide-to-spectrum fragment-intensity prediction is a potentially novel application. The authors convert peptide sequences into theoretical fragment-ion ladders and adapt the spectrum encoder for intensity prediction. This is a creative way to reuse the model, although the practical benefit over existing sequence-based approaches is limited.

Critical weaknesses

1. Insufficient motivation for the pretraining objective and unclear significance of the scaling law

My overall impression is that the authors pursued scaling before finding a suitable pretraining objective.

Motivation and Figure 1. Figure 1 presents a scaling law primarily measured by masked-intensity reconstruction loss, which is not particularly interesting on its own. Why should we care that a much larger model can reduce reconstruction loss unless this is shown to correlate with performance on meaningful downstream tasks? De novo peptide sequencing is an obvious example: it has seen a surge of ML research in recent years and is increasingly being used in experimental studies, with important applications in metaproteomics, immunopeptidomics, and identifying sequence variants that are difficult to account for in sequence databases. This is a major direction in the field of ML for peptide MS/MS, yet the authors do not evaluate it at all. Spectrum retrieval is another relevant task, with applications in spectral-library searching, but their results here are particularly unconvincing, as discussed below. While the authors evaluate other useful downstream tasks, they do not convincingly establish that scaling their pretraining objective translates into improved downstream performance.

I can concede that masked m/z reconstruction is problematic, even if that was not brought up by the authors. For example, multiple plausible fragments may be missing between observed peaks. How should a model be penalized for predicting a plausible fragment that is absent from the target spectrum? I presume this is why the authors resort to masked-intensity prediction, although the choice is poorly motivated. Intensities carry substantially less chemically identifying information than m/z coordinates, and avoiding the ambiguity of m/z imputation does not automatically make intensity reconstruction a good pretraining objective.

Nevertheless, the authors push ahead with scaling this objective, spending substantial compute without first establishing that it produces useful representations. I find the motivation and validation of the pretraining objective insufficient.

The abstract asks whether larger models can "genuinely understand mass spectra". What does it mean to "understand" spectra? This is very vague language. A stronger motivation would define this in terms of learning representations useful for concrete downstream tasks.

Similarly, Section 3 states the objective but provides little justification for this choice or why intensity reconstruction should be an effective representation-learning objective.

The introduction frames two main questions: "Does scale help?" and "Is absolute mass necessary?" Why these two questions? A more fundamental question seems to be: What is a suitable self-supervised pretraining task for MS/MS spectra in the first place?

Relative-mass-only encoding. The second major research question concerns whether absolute m/z information is necessary. The authors note that previous models use absolute m/z, sometimes supplemented with pairwise mass differences, and propose removing absolute m/z entirely.

However, there is no ablation comparing otherwise identical models with and without absolute m/z information.

The framing also seems somewhat backward. A reasonable question would be whether explicitly providing higher-order features such as mass differences improves performance. This was already investigated in the cited prior work (Lapin et al., 2025; Bushuiev et al., 2025), with Lapin et al. reporting that the benefit narrows on larger training corpora.

Removing absolute m/z could be a somewhat interesting architectural choice, but its benefit is not demonstrated. The authors later argue that their encoder is invariant to global shifts in m/z and must therefore learn relationships rather than memorize absolute positions. This could motivate the design, but it remains unclear whether this inductive bias improves performance or generalization without an ablation study.

2. Unconvincing downstream scaling and substantial evaluation limitations

Figure 2(a,b): Denoising. As I understand it, this is a per-peak classification task distinguishing genuine fragment peaks from noise.

Performance saturates relatively early and shows no improvement beyond the 200M model, despite continued reductions in pretraining reconstruction loss. This is particularly concerning because denoising is closely related to the pretraining objective, limiting what these results demonstrate about general-purpose representations. If scaling provides little additional benefit on this task, it is unclear whether it should be expected to help on more challenging tasks such as de novo sequencing, which is not evaluated, or retrieval.

There are also serious evaluation issues:

  • Appendix J states that the denoising recipe was selected using test AUROC from a grid of 216 hyperparameter configurations at 50M. The test set was therefore used for model selection rather than reserved for independent evaluation.
  • The authors report that reloaded checkpoints produce AUROC values approximately 0.014 lower than their logged evaluations, an unresolved discrepancy that further undermines confidence in the results.

Figure 2(c): Retrieval. The figure presents retrieval scaling without baseline references, making the absolute performance difficult to interpret. The 50M through 200M models achieve almost identical MAP@R, with only the 400M model showing a noticeable improvement. This could indicate some scaling benefit, but without the GLEAMS or cosine baselines, introduced only in Figure 3, the results are difficult to assess.

Figure 2(d,e): Synthetic chimera decomposition. These panels evaluate chimera detection and peak assignment on purely synthetic mixtures generated from the same MSConsensus corpus used for pretraining. Synthetic mixtures may be useful for training or controlled evaluation, but this alone establishes little about performance on real chimeric spectra.

The detection task is also nearly saturated, with even the random-backbone baseline achieving approximately 0.95 AUROC, while peak-assignment F1 saturates at 200M. Furthermore, the training procedure includes 20k "half-real" examples, without clearly explaining what these are or how they were constructed. The task itself is also poorly introduced in the main text; the detection and two-slot peak-assignment heads are only described in Appendix E.

Figure 2(f): Real HCT116 decomposition. This is potentially more interesting, but the evaluation is quite limited. It uses only 1,376 search-corroborated peptide pairs. The ground-truth peak assignments are search-derived pseudo-labels rather than independently verified annotations, and Appendix R acknowledges that pair-level FDR is not controlled. The construction of these labels and the acquisition setup are insufficiently explained.

Furthermore, the reported advantage over the random-backbone control disappears on an external HEK dataset at 200M (Appendix R), raising additional concerns about generalization.

Overall, Figure 2 provides limited evidence for consistent downstream scaling. Several tasks saturate early, the retrieval trend is unclear, and the decomposition results have important generalization limitations.

Figure 3(a): Retrieval. The results here are particularly concerning. Iona performs very well on the in-distribution benchmark but suffers severe performance degradation on several external datasets.

Their supervised contrastively fine-tuned model falls substantially behind the older GLEAMS model and even simple binned cosine similarity. On low-resolution HEK293, the final fine-tuned Iona model achieves only 0.170 MAP@R, compared with 0.530 for GLEAMS and 0.554 for binned cosine. The first-stage Iona model performs even worse, at 0.017.

On the nine-species yeast benchmark, a common evaluation dataset, the fine-tuned Iona model is again substantially behind both baselines. On the 20k yeast subset, it achieves 0.600 MAP@R versus 0.770 for GLEAMS and 0.916 for binned cosine.

In Section 4.4: Comparison with specialized models, the authors write:

"Contrastive specialisation of Iona-400M reaches MAP@R 0.868 on its own distribution, far above GLEAMS [...] (0.646) and cosine [...] (0.729)."

The authors emphasize their strong in-distribution retrieval performance despite substantial failures on several external benchmarks. This is a fundamental issue in evaluating machine learning models, particularly for MS/MS data, where multiple highly similar spectra from the same peptides and charge states are common. Good performance on a held-out test set drawn from the supervised fine-tuning distribution does not establish generalization across acquisition conditions, especially when performance degrades so severely under distribution shift. The fact that Iona falls well behind GLEAMS and binned cosine on multiple external datasets should be a central finding, rather than a secondary qualification to the in-distribution results.

At the very least, the authors should investigate why their supervised contrastive fine-tuning performs so poorly compared with a model from 2022 and even non-learned similarity measures.

One possible explanation is that masked-intensity reconstruction is poorly suited to learning transferable spectrum-level representations. The pretraining objective operates at the peak level and does not directly optimize spectrum-level embeddings. However, the experiments do not distinguish this possibility from shortcomings of the contrastive fine-tuning procedure.

These results substantially weaken the evidence that the proposed pretraining approach learns transferable weights.

Figure 3(b): PSM rescoring. The rescoring pipeline is considerably more complicated than simply applying the pretrained spectrum encoder. It first uses supervised contrastive learning based on peptide identities, then trains an additional peptide Transformer to reproduce the spectrum encoder's embeddings from annotated peptide sequences and charges. Similarities between the two embeddings are subsequently used as features for PSM rescoring.

This second model is barely described. Appendix E essentially states only that it is a Transformer trained by MSE against the spectrum embeddings, without specifying its architecture, parameter count, or training hyperparameters. This is a serious reproducibility issue for a model that is essential to one of the paper's headline results.

Lack of a baseline: the final system relies on supervised contrastive training using peptide identities, followed by training an additional peptide encoder to reproduce the spectrum embeddings. A natural baseline would be to train the spectrum encoder from scratch using the same supervised contrastive objective, then train the peptide student and evaluate the complete rescoring pipeline identically. Without this comparison, it is unclear how much of the final performance comes from the proposed self-supervised pretraining rather than the subsequent supervised training. The shuffled-embedding control does not answer this question.

The headline PSM rescoring result is obtained on 18 HCT116 mass spectrometry runs, where adding Iona embedding features to MSFragger-based rescoring yields 20,763 additional PSMs at 1% FDR over a shuffled-embedding control. However, when adding Iona to the stronger MS2Rescore pipeline, including MS2PIP and DeepLC, the net improvement is only 251 PSMs on a separate eight-run dataset (Appendix M). The authors do not evaluate the full MS2Rescore pipeline on the original 18-run dataset. Consequently, it remains unclear whether the headline gains would persist against a stronger established baseline, or how much of the difference is explained by the datasets.

Fragment-intensity prediction and Prosit. Repurposing a pretrained spectrum encoder for peptide-to-spectrum intensity prediction is a potentially novel aspect of this work. However, the adaptation is rather convoluted. The authors convert peptide sequences into theoretical b/y-ion masses, assign peptide-agnostic prior intensities, and pass these through Iona with a newly trained prediction head (Appendix E).

This is a creative way to reuse the spectrum encoder, but why take this detour instead of directly encoding the peptide sequence, as existing methods do? The authors do not establish any advantage to this approach.

The results approximately match the older Prosit RNN but fall behind Prosit Transformer. More importantly, without a scratch-initialized Iona baseline, it is impossible to tell whether the pretrained representations contribute meaningfully to the final performance. Task-specific adaptation is perfectly reasonable for a foundation model, but the authors need to demonstrate that the pretraining is actually useful for this task.

Overall assessment. The paper's central weakness is that the authors attempt to establish compute-optimal scaling without first convincingly establishing the suitability of their pretraining objective. Additionally, it is lacking in novelty and clarity.

The scaling law concerns masked-intensity reconstruction loss, while the downstream experiments show inconsistent scaling and substantial generalization problems. Several evaluations also have methodological limitations, including test-set-based hyperparameter selection, unresolved reproducibility discrepancies, incomplete controls, and limited independent validation.

There are some promising application results, but the paper does not convincingly establish that continued scaling of this particular pretraining objective leads to meaningful, generalizable downstream improvements, or that its scaling law provides useful guidance for compute-optimal foundation-model development.

Questions for authors
  1. Pretraining objective and relevance: What motivated the choice of masked-intensity reconstruction as the pretraining objective? What evidence supports its suitability for learning useful, transferable MS/MS representations? In particular, can the authors demonstrate that improvements in reconstruction loss predict performance on meaningful downstream tasks, or provide comparisons against alternative self-supervised objectives?

  2. Denoising evaluation: Appendix J states that the denoising recipe was selected using test AUROC. Can the authors provide results on an independent test set and explain the discrepancy between logged and reloaded checkpoint performance?

  3. Chimera decomposition: How exactly are the synthetic chimeric spectra generated, and how are their peak-level ground-truth labels assigned, particularly when fragment peaks overlap? What does "half-real" mean, and how are these examples constructed? Similarly, how are the search-derived peak assignments obtained for the real HCT116 spectra, what is the acquisition setup, and how reliable are these pseudo-labels? These are essential details for assessing the validity of the decomposition experiments.

  4. "PSM rescoring and pretraining controls: Can the authors provide the architecture, parameter count, and full training details of the peptide encoder? More importantly, can they evaluate a scratch-initialized spectrum encoder under the same complete downstream training and rescoring pipeline, to isolate the contribution of masked-intensity pretraining? The same concern applies to fragment-intensity prediction."

  5. Relative-mass encoding: Can the authors provide an ablation study comparing relative-only encoding against architectures that also include absolute m/z information? Without this, it is unclear whether removing absolute mass offers any practical benefit.

1 comment

“What motivated the choice of masked-intensity reconstruction as the pretraining objective? [...]”

We have revised the manuscript to motivate the objective (Sections 2 and 3.2). Masked m/z prediction is ill-posed, since valid fragments are often absent, whereas masked intensity keeps m/z visible, so the target is well defined. Because mass enters only through pairwise differences, predicting intensity requires identifying chemically related peaks, which the model learns without annotation (Section 4.2). We did not compare alternative objectives at matched compute (Section 5), but reconstruction loss predicts downstream performance on two new benchmarks (below).

“De novo peptide sequencing is an obvious example [...]”

We originally focused on the established database-search pipeline. De novo sequencing alone is not a clean test of what a foundation model has learned, since encoder and decoder are trained jointly on PSMs, but it is a strong test of transfer, so we have added it (Section 4.4, Appendix T). Under identical one-epoch training, pretraining raises yeast peptide recall from 0.398 to 0.675 (Table 2), and pretrained Iona-50M exceeds Casanovo v5 in mean peptide recall across nine species (0.683 vs. 0.620; Table 26).

“[...] there is no ablation comparing otherwise identical models with and without absolute m/z information.”

Prior work (Lapin et al., 2025; Bushuiev et al., 2026) asks whether adding mass differences to an absolute-mass encoder helps. We ask whether absolute m/z is needed at all, since removing it makes the encoder invariant to calibration offsets. In a matched ablation (Section 4.3, Appendix S), the absolute variant fits pretraining slightly better but loses up to 15% AUROC under constant offsets, where Iona is unaffected, and degrades 5–9× more under ppm drift (Table 25).

“Overall, Figure 2 provides limited evidence for consistent downstream scaling [...]”

We have added two real-data benchmarks (Section 4.4, Appendix U). From 25M to 400M, DDA chimericity AUROC rises from 0.832 to 0.872 and DIA precursor AP from 0.698 to 0.748, both above fine-tuned CasanovoTL, and both track pretraining loss (r = −0.97 and −0.98). A randomly initialized encoder is no better than no encoder. The tasks that saturate at 200M are closest to the objective, and pretraining loss itself barely changes from 200M to 400M (Table 1).

“[...] the denoising recipe was selected using test AUROC [...] reloaded checkpoints produce AUROC values approximately 0.014 lower [...]”

Ranking the grid by validation AUROC selects the same configuration apart from a narrower head (Appendix J). The reload discrepancy was caused by a DeepSpeed checkpoint bug, was mistakenly left in the submission, and has been removed.

“How exactly are the synthetic chimeric spectra generated [...] how reliable are these pseudo-labels?”

Synthetic generation, peak labels, half-real training, the HCT116 setup and the pseudo-label limitations are now in Appendix R.

“Iona performs very well on the in-distribution benchmark but suffers severe performance degradation on several external datasets.”

We agree and have revised Sections 4.5 and 5. Our results point to the contrastive fine-tuning, since the frozen Iona-400M encoder reaches MAP@R 0.709 on unseen yeast, above every fine-tuned model (Appendix H). Applied to small molecules, Iona outperforms DreaMS on library search, including cross-instrument and novel-molecule settings (Section 4.5, Appendix V).

“Can the authors provide the architecture [...] of the peptide encoder? [...] evaluate a scratch-initialized spectrum encoder [...]”

Peptide-student details are in Appendix E, and the random-initialization control is in Section 4.4 and Figure 9. The 8-run MS2Rescore comparison is in Table 16; the 18-run comparison was not feasible within the rebuttal period.

“[...] without a scratch-initialized Iona baseline [...]”

Pretraining raises the median spectral angle from 0.83 to 0.91 (Table 18). The task tests whether the encoder has learned fragmentation chemistry from mass relationships alone, not whether it replaces Prosit.

“Additionally, it is lacking in novelty and clarity.”

We respectfully disagree. The reviewer notes that establishing how pretraining scale affects downstream performance “could be valuable for this relatively understudied modality.” This is what our work does. To our knowledge, it is the first scaling study of pretrained spectrum models (Appendix B). It shows that loss follows a power law, that pretraining loss predicts real-data performance, that gains come from pretraining rather than architecture, that absolute m/z is unnecessary, and that the recipe transfers to small molecules. We ask the reviewer to reassess the paper in light of these results.

Reviewer_cYhHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents Iona, a family of self-supervised mass spectrometry foundation models ranging from 25M to 400M parameters, pretrained on 100 million consensus MS/MS spectra using a masked intensity reconstruction objective. The authors investigate scaling laws for mass spectrometry models and propose a relative-mass-only encoder that uses pairwise m/z differences instead of absolute m/z values. The experiments cover several downstream tasks, including denoising, spectrum retrieval, chimeric-spectrum decomposition, and peptide identification.

Critical strengths
  • The paper investigates an interesting and important question about scaling laws for mass spectrometry foundation models. The systematic experiments across five model sizes provide useful insights into the effect of model scale.
  • The experiments cover multiple downstream tasks, and the comparisons with specialized models and random-initialization controls help demonstrate the effectiveness of pretraining.
Critical weaknesses
  • The paper appears to be AI-generated, as it presents a large number of experimental results without sufficient in-depth analysis or discussion, particularly in Sections 4.3 and 4.4. Moreover, in Appendix I, the authors refer to "the final checkpoint of Iona-50M (data/delta_bias.csv)," which confusingly describes a CSV file as a model checkpoint. This kind of imprecise wording further raises concerns about AI-generated writing and insufficient manual verification.

  • In Section 3.1, the model architecture is not clearly presented. The paper mainly describes the Fourier features applied to pairwise mass differences, but the overall encoder architecture and how the 512 input peaks are processed are difficult to follow. A more detailed architecture illustration would be helpful.

  • In Section 4.1, the authors fit Eq. (4) and conclude that the compute-optimal model size is beyond 400M parameters. However, it is unclear why this particular function is chosen and how reliable the extrapolation is. What is the estimated optimal model size under the current compute budget? If the optimal model size can be determined, are there any experiments to verify that it is indeed optimal under the same compute budget?

  • The relative-mass-only design is a major focus of this paper. However, the authors do not compare it with an identical encoder using absolute m/z or a combination of absolute and relative m/z. Although the authors acknowledge this limitation in Section 5, I strongly suggest including these ablation experiments to better justify the proposed design. In particular, it remains unclear whether removing absolute m/z provides any advantage over combining both types of information.

  • In Lines 157–161, the authors mention that the training hyperparameters are fixed using the 50M model and reused for all model sizes. Is it reasonable to use identical hyperparameters for models ranging from 25M to 400M parameters? This choice may affect the scaling law conclusions and should be better justified.

  • In Section 3.3, the description of Contrastive Specialisation is difficult to follow. In particular, the relationship between supervised contrastive learning, the spectrum teacher, and the peptide student is not clearly explained. More implementation details would be helpful.

  • The authors claim that the same scaling behavior is expected to hold for metabolomics and other mass spectrometry applications. However, all pretraining and downstream experiments are conducted on proteomics data. I suggest including a simple transfer experiment on small-molecule mass spectra to support this claim.

  • The authors claim that the code and checkpoints are publicly available, but the provided anonymous repository link could not be accessed.

Questions for authors

From Table 1, larger models generally achieve better performance on downstream tasks, which is expected and does not provide much new insight. Moreover, the choice of the fitting function in Eq. (4) lacks sufficient justification. Could the authors explain why this particular function is used? Additionally, the relative-mass-only design has not been shown to outperform approaches using absolute m/z or a combination of absolute and relative m/z. What is the advantage of the proposed design over these alternatives?

1 comment

“In Section 4.1, the authors fit Eq. (4) [...] it is unclear why this particular function is chosen and how reliable the extrapolation is. [...]”

We thank the reviewer for this question. Eq. (4) is the additive parametric form of Hoffmann et al. (2022, Approach 3), treating loss as separable power-law contributions from model size and training exposure. We chose it because it is the standard form in the scaling-law literature, making our exponents comparable across modalities. The fit describes our data closely (log-space R² = 0.99, mean absolute error 1.1% over 40 checkpoints). Figure 1 also supports this conclusion independently of Eq. (4), since each larger model overtakes the smaller ones as compute grows and 400M lies on the frontier at the largest budgets we trained.

At the final 400M budget (3.4 × 10^20 FLOPs), minimizing Eq. (4) under C = 6ND gives N_opt ≈ 810M, about twice our largest model. Verifying this requires an IsoFLOP sweep, which was not feasible within the rebuttal period (Section 5); caveats of the fit are in Appendix G.

“[...] What is the advantage of the proposed design over these alternatives?”

We have added a matched 25M ablation in which mass enters only through absolute m/z (Section 4.3, Appendix S). The absolute variant has slightly lower pretraining loss (0.0658 vs. 0.0688, single seed). This gap is small, less than half the improvement from doubling model size. The main advantage is robustness to mass calibration error, which is common in practice (Appendix P). Under test-time drift on denoising (Table 25), the relative-only encoder is exactly unaffected by constant offsets, where the absolute variant loses up to 15% AUROC, and it degrades 5–9× less under ppm drift. A combined absolute + relative encoder is not shift-invariant by construction; we leave it to future work (Section 5).

“The paper appears to be AI-generated [...]”

We thank the reviewer for this feedback. As disclosed in our AI Use Statement, we used language model tools to edit and polish the text. All experiments, analyses and results were designed and conducted by the authors. We take full responsibility for the imprecise wording, have corrected it (Appendix I), and have carefully proofread the revision.

“In Section 3.1, the model architecture is not clearly presented [...]”

We have added an architecture illustration (Figure 5, Appendix A); per-size configurations are in Table 6 (Appendix C).

“[...] Is it reasonable to use identical hyperparameters for models ranging from 25M to 400M parameters? [...]”

Fixing hyperparameters tuned at small scale is common in scaling studies (Hoffmann et al., 2022; Kaplan et al., 2020), and per-size sweeps were not feasible within our compute budget. Because ours were selected at 50M, per-size tuning could only improve the other sizes, so their reported losses are upper bounds. This would likely strengthen our conclusion that loss has not saturated and the compute-optimal size lies beyond 400M. This discussion is now in Section 3.2, Appendix G and the Limitations.

“In Section 3.3, the description of Contrastive Specialisation is difficult to follow [...]”

We have rewritten this description (Section 4.4, Appendix E). The spectrum encoder is fine-tuned with a supervised contrastive loss so that spectra of the same peptide embed together, and a separate 4.8M-parameter peptide encoder (the student) is trained to map sequences into this frozen embedding space.

“I suggest including a simple transfer experiment on small-molecule mass spectra to support this claim.”

We have added this experiment (Section 4.5, Appendix V). Pretrained on 25.2M small-molecule spectra, Iona outperforms DreaMS on spectral library search, and even the proteomics-pretrained encoder transfers.

“The provided anonymous repository link could not be accessed.”

We apologize for this; the link has been updated and is now accessible (Reproducibility Statement).

Comments on the paper

1 comment

We thank all reviewers for their feedback, which has substantially strengthened the paper. We respectfully but firmly disagree with Reviewer 8g4q’s assessment that the paper offers little or nothing new. To our knowledge, this is the first scaling-law study of mass spectrometry models. Across 33 published spectrum models, none varies model size beyond 15M parameters or fits a compute–loss relation (Appendix B). Establishing how scale affects this modality is, we believe, a contribution in its own right.

The main concerns raised are now addressed with new experiments in the revised manuscript:

  • Absolute-mass ablation (all reviewers): a matched ablation showing the relative-only design trades a small loss in pretraining fit for robustness to calibration error (Section 4.3).
  • De novo sequencing: pretrained Iona-50M exceeds Casanovo v5 in mean peptide recall (Section 4.4).
  • Downstream scaling: two new real-data benchmarks where performance improves monotonically to 400M and tracks pretraining loss (r = −0.97 and −0.98) (Section 4.4).
  • Random-initialization controls: these show the gains come from pretraining (Section 4.4).
  • Small-molecule transfer: Iona outperforms DreaMS on spectral library search (Section 4.5), confirming our claim that the recipe extends beyond proteomics.

We ask the reviewers to reassess the submission in light of these results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.