- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents a family of transformer-based foundation models (FMs) for microbiome data, named Waypoint, with sizes ranging from 6 to 170M parameters. The authors also present a benchmark of downstream tasks containing classification and regression tasks. The FMs are pretrained on ~540k microbiome compositional samples extracted from the MGnify database. The paper compares the proposed FMs at different model sizes to traditional ML-based models and an existing microbiome FMs (MGM). The results show that Waypoint models outperform the MGM model and are competitive against traditional models like XGBoost, even outperforming it for larger microbiome datasets (10k+ samples).
Critical strengths
- This work presents a foundation model for microbiome compositional data which are traditionally represented as a normalized (CLR, for example) vector of the read counts per taxon. Embeddings from the FMs present a different way to represent microbiome data.
- The paper builds on ideas from other recent papers, such as Zhang2026, and extend it by bringing in more pretraining data and the creation of a benchmark of downstream microbiome predictive tasks.
- I appreciate the amount of information in supplementary material that help me better assess the work.
Critical weaknesses
-
As stated earlier, the paper has limited methodological novelty. One of the few claimed methodological contribution is the “fallback” taxa selection in the tokenizer; however, the authors do not present a systematic evaluation of its effective performance contribution to the model.
-
The data utilized for pretraining is a collection from many microbiome studies present in Mgnify. The paper does not discuss/address the bias present in individual datasets, which is a well-known characteristic of microbiome data. Each study follows an independent protocol, uses different sequencing platforms, distinct taxonomic classification pipelines, different quality and sequence filtering steps, etc; all of which have a strong influence in the final compositional abundance profile of the samples. Even though MGnify attempts to harmonizes the computational processing of data across studies, it does not guarantee that the resulting profiles are free of batch effects, given that uniformized computational processing cannot undo the differences introduced in the wetlab and data generation. Additional to that, the authors pulled data from different versions of MGnify, and different versions use different processing steps, classifier, and reference databases (as described in their website/papers). A simple check would be to plot the samples embedding and hue it by their MGnify version.
-
Still related to the batch effect issues mentioned above, the order of the taxa in the input sequence, which is the only abundance information the model receives, is defined by z-scores computed with the mean and standard deviation of each taxon over the entire pretraining corpus (~485k samples). However, this corpus mixes 16S amplicon, shotgun and assembly data, MGnify pipeline versions v1.0 to v5.0 (with different classifiers and reference databases, etc), and over thousands of studies with different extraction protocols, primers and sequencing depths, all of which shift the relative abundances systematically by source. Then, the pooled mean and standard deviation are mixture statistics, and the z-score of a taxon partly measures how far a sample is from the average of the sources that dominate Atlas, not only its biology. This suggests that the token order, and consequently the next-token prediction objective, may encode technical provenance. The problem gets worse in the downstream tasks, where datasets processed with yet other pipelines are reordered using statistics they were never drawn from. This adds a second distribution shift on top of the vocabulary mismatch the authors already discuss . The paper does not discuss this, and it affects the interpretation of the main results. The biome structure observed in the embeddings in Figures 1C and 1D could partly reflect study, modality or pipeline. Additionally, the cross-domain transfer and scaling conclusions cannot be separated from technical confounding. The 10k crossover is also confounded, since the MGnify tasks (1 and 2) share the pretraining statistics while the other datasets do not. Again, simple checks would be to color the embeddings by modality, pipeline version and study. Other important analysis would be to compare the z-score ordering against a plain within-sample abundance ranking. Finally, the authors did not make it clear whether zeros are included when computing mu_t and sigma_t.
-
The results presented do not show clear benefit of the proposed FMs when compared to XGBoost. The authors do not present a proper hypothesis test analysis. Additionally, the paper presents a series of models but only the best one, selected after the fact (after seeing the performance on the downstream tasks), is compared against XGBoost on different downstream task's sample size. It shows that the proposed 45M version of their FM beats XGboost on a datasets with 10k+ samples, which is impractical for most of the microbiome studies, where only hundreds of samples are usually available due to the cost and time to collect microbiome samples. See [1] for example, where all the 84 microbiome studies collected have less than 2k samples, with the large majority with a few hundreds of samples. Also, as mentioned above, from the 3 tasks with 10k+ samples, tasks 1 and 2 are from MGnify, so we cannot be sure whether the improvement over XGBoost for these tasks are due to sample size or due to be drawn from the pretraining dataset.
-
The paper does not present clearly what’s the pretraining task until you read the supplementary material. The main paper should show how a microbiome sample is represented as sequence of tokens so it can be fed to a transformer-based model.
Minor points:
- The fact that an approach is comparatively underexplored is not, by itself, a sufficient motivation. The paper should identify specific limitations of existing methods and explain why the proposed approach is suited to addressing them. This motivation should be grounded in the characteristics of microbiome data. For example, how could the transformer architecture or training objective account for the compositional nature of these data?
- The relatively extreme low MLP results seems to indicate that something is wrong with its training. I suggest the authors to carefully revise the MLP experiments.
Refs: [1] Car Reen Kok, Nisha J Mulakken, James B Thissen, Jose Manuel Martí, Ryan Lee, Jacob B Trainer, Andre R Goncalves, Hiranmayi Ranganathan, Aram Avila-Herrera, Crystal J Jaing, Nicholas A Be, Meta2DB: curated shotgun metagenomic feature sets and metadata for health state prediction, Bioinformatics, Volume 42, Issue 7, July 2026, btag422, https://doi.org/10.1093/bioinformatics/btag422
Questions for authors
I don't have additional questions.