acceptodds
Under review as a conference paper at ICLR 2027

TaxoMFM: A Taxonomy-Aware Foundation Model for the Human Gut Microbiome

Abstract

Existing gut microbiome foundation models typically use flat taxon representations, limiting their ability to exploit known taxonomic hierarchies. We introduce TaxoMFM, a microbiome foundation model that jointly models four taxonomic ranks spanning species, genus, family, and order. TaxoMFM encodes taxonomic relationships through relation-biased self-attention and performs multiscale compositional reconstruction and hierarchical composition learning with a hierarchical compositional decoder. We further construct TaxoGut-HPP, comprising 100,806 gut microbiome profiles from 231 source cohorts and covering 40 classification and eight regression tasks. TaxoMFM achieves the strongest overall performance across disease, gut health, host trait, and liver ultrasound phenotype prediction. In cross-cohort zero-shot transfer, where target cohorts are excluded from both pretraining and downstream adaptation, TaxoMFM achieves AUROCs of 0.838, 0.736, and 0.722 for colorectal cancer, type 2 diabetes, and inflammatory bowel disease, respectively. Further analyses show that TaxoMFM recovers community changes across taxonomic scales, while preserving community information and producing consistent representations under database updates and species-vocabulary shifts. Our code is available at https://anonymous.4open.science/r/TaxoMFM-F16D/.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
microbiome foundation models, representation learning, computational biology

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 4.00 over 3 reviews, above 99% of the others.
Reviewer_rKPDOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces TaxoMFM, a foundation model for human gut microbiome profiling that leverages microbial taxonomic hierarchies through self-supervised learning. Unlike conventional approaches that represent microbial species independently, TaxoMFM integrates multiple taxonomic levels and incorporates attention biases based on taxonomic relationships. The model also introduces a hierarchical compositional decoder to reconstruct microbial abundances across taxonomic levels. During pretraining, it combines multiscale composition reconstruction with hierarchical composition learning to capture both microbial abundances and relative compositions within higher taxonomic groups. The authors construct TaxoGut-HPP, a comprehensive microbiome dataset collected from diverse cohorts, and evaluate the model on various classification and regression tasks. TaxoMFM demonstrates strong predictive performance compared with existing methods, while ablation studies highlight the contributions of taxonomic structure and hierarchical learning objectives.

Critical strengths
  1. TaxoMFM effectively integrates taxonomic hierarchy into both its attention mechanism and hierarchical compositional decoder, rather than treating taxonomy as an additional input feature. Learning microbial abundances and relative compositions within taxonomic groups is well suited to the characteristics of microbiome data.

  2. The paper evaluates TaxoMFM across diverse prediction tasks, including disease classification, gut health symptoms, host characteristics, and liver health indicators. The results demonstrate consistent improvements over existing methods across various tasks.

Critical weaknesses
  1. TaxoMFM is pretrained on 100,806 profiles, substantially more than BiomeGPT (13,524) or GUT-FORMer (21,024). While the ablations support the proposed design, they do not rule out the possibility that part of the improvement over existing models comes from the larger pretraining corpus. Comparisons under matched data and training budgets would strengthen the architectural claims.

  2. The cross-cohort evaluation is limited to three target cohorts, including only 130 samples for IBD. The improvement on IBD is also relatively modest (0.722 vs. 0.693 AUROC). Without uncertainty estimates or additional cohort combinations, it is difficult to judge how reliably these gains generalize.

  3. The main novelty seems to lie in combining existing ideas rather than introducing a substantially new modeling approach. While the empirical results are strong, the methodological contribution beyond prior taxonomy-aware models is not entirely clear.

Questions for authors
  1. Could the authors compare TaxoMFM with existing foundation models pretrained on the same data, to better isolate the contribution of the proposed architecture?

  2. For the IBD cross-cohort result (0.722 vs 0.693, n=130), is the improvement statistically meaningful? Could the authors report bootstrap confidence intervals for Table 5?

  3. In Table 8, TaxoMFM retains 81.44% of the abundance while BiomeGPT retains only 45.79%. How much of the improved vocabulary robustness is attributable to ancestor routing rather than the learned representations?

5 comments
AuthorsOpenReview ↗

Response to Reviewer (Part 1/3)

  1. W1: TaxoMFM is pretrained on 100,806 profiles, substantially more than BiomeGPT (13,524) or GUT-FORMer (21,024). While the ablations support the proposed design, they do not rule out the possibility that part of the improvement over existing models comes from the larger pretraining corpus. Comparisons under matched data and training budgets would strengthen the architectural claims.

Thank you for this comment. All foundation models in Tables 2, 3 and A3–A8 are pretrained from scratch on the same TaxoGut-HPP samples as TaxoMFM. The pretraining data size is therefore the same for every model in these comparisons. Each foundation model keeps the architecture and pretraining objective of its original paper. BiomeGPT has no public code, so we implemented it from the description in its paper. On each downstream task, all models use the same participant splits and metrics. The numbers 13,524 and 21,024 are the pretraining sizes reported in the original BiomeGPT and GUT-FORMer papers. Table 1 lists the data scale of each model's original paper. We will note this in the caption of Table 1 and state the pretraining data of all foundation models in Section 4.1.

We agree that the pretraining budget matters when comparing models. On a single RTX PRO 6000 GPU and the same dataset, we estimate the pretraining time at 36.6, 33.8, 38.1 and 53.8 GPU-hours for TaxoMFM, BiomeGPT, MGM and Waypoint-45M. With a budget close to BiomeGPT and MGM, TaxoMFM ranks first on 46 of the 48 downstream tasks and achieves the highest AUROC on all three zero-shot cross-cohort tasks (Table 5). It also leads in multiscale perturbation recovery, rare-species abundance recovery and cross-vocabulary sample matching (Tables 6–8). We will list the pretraining time of each model in Appendix D.

  1. W2: The cross-cohort evaluation is limited to three target cohorts, including only 130 samples for IBD. The improvement on IBD is also relatively modest (0.722 vs. 0.693 AUROC). Without uncertainty estimates or additional cohort combinations, it is difficult to judge how reliably these gains generalize.

Thank you for this suggestion. We add two independent IBD target cohorts: Cork (n = 87) and UCSD (n = 197). As in Table 5, the samples of these cohorts are excluded from both pretraining and downstream training. We keep the source-cohort training setup of the original experiment. Values are AUROC (95% CI).

ModelCork (n = 87)UCSD (n = 197)
Random Forest0.713 (0.611–0.813)0.669 (0.563–0.772)
MIOSTONE0.703 (0.589–0.807)0.652 (0.552–0.757)
PhyloGCNE0.651 (0.527–0.771)0.629 (0.531–0.730)
MGM0.769 (0.668–0.858)0.688 (0.577–0.789)
BiomeGPT0.792 (0.710–0.881)0.667 (0.555–0.771)
Species-only0.762 (0.662–0.861)0.659 (0.561–0.753)
TaxoMFM0.846 (0.762–0.923)0.727 (0.629–0.815)

TaxoMFM achieves the highest AUROC on both new cohorts. On Cork, its AUROC is 0.846 (0.762–0.923), 0.054 above the strongest baseline. On UCSD, its AUROC is 0.727 (0.629–0.815), 0.039 above the strongest baseline. Together with the gain of 0.029 on the original IBD target cohort, TaxoMFM outperforms the best baseline on all three independent IBD cohorts.

  1. W3: The main novelty seems to lie in combining existing ideas rather than introducing a substantially new modeling approach. While the empirical results are strong, the methodological contribution beyond prior taxonomy-aware models is not entirely clear.

Thank you for raising this point. TaxoMFM differs from prior taxonomy-aware models in where the taxonomy is used. Models such as MIOSTONE use the taxonomy inside a supervised predictor. They connect taxa across ranks, aggregate features rank by rank, and are trained separately for each phenotype. TaxoMFM uses the taxonomy to define the self-supervised pretraining objective. A single pretrained representation then serves all downstream tasks. This representation separates changes in the total abundance of a lineage from changes in its internal composition. This separation matters for studies of disease and host phenotypes. For example, when the relative abundance of a species increases, the whole lineage may have expanded. Alternatively, the lineage total may be stable while the species takes a larger share within it.

During pretraining, we mask the abundances of some species or of whole taxonomic branches, and reconstruct the community from a single sample representation. The multiscale reconstruction loss (Eq. 8) requires the model to recover the abundance distribution at the species, genus, family and order ranks. This preserves the share of each lineage in the whole community. The hierarchical composition loss (Eq. 9) directly supervises how abundance is divided within each parent. The model therefore captures shifts among members even when the lineage total is stable. The controlled perturbation experiment in Table 6 shows that TaxoMFM recovers both types of change more accurately from partial observations.

AuthorsOpenReview ↗

Response to Reviewer (Part 2/3)

  1. Q1: Could the authors compare TaxoMFM with existing foundation models pretrained on the same data, to better isolate the contribution of the proposed architecture?

Thank you for this suggestion. This is the setting of our comparison. TaxoMFM and all other foundation models are pretrained from scratch on the same set of TaxoGut-HPP samples, as described in our reply to W1. We will state this explicitly in Section 4.1. We chose TaxoGut-HPP because it provides large-scale pretraining data and covers 40 classification and 8 regression tasks across disease, gut health and continuous physiological traits.

Under this shared setting, TaxoMFM outperforms the foundation models without explicit taxonomic hierarchy on the classification and regression tasks in Tables 2 and 3. For example, on IBD classification in Table 2, TaxoMFM reaches an AUROC of 0.985, against 0.966 for the best baseline, GUT-FORMer. On the liver attenuation coefficient in Table 3, TaxoMFM reaches a Spearman ρ\rho of 0.347, against 0.329 for BiomeGPT.

Table 4 further tests the contribution of the taxonomic hierarchy through architectural ablation. With the same pretraining data, removing the genus, family and order nodes lowers the mean AUROC on colorectal cancer, IBD and type 2 diabetes from 0.957 to 0.922. This supports the contribution of explicit hierarchy modeling to downstream prediction.

  1. Q2: For the IBD cross-cohort result (0.722 vs 0.693, n=130), is the improvement statistically meaningful? Could the authors report bootstrap confidence intervals for Table 5?

Thank you for this question. As you note in W2, the IBD target cohort has only 130 samples, so a test on this cohort alone has limited power. We therefore add two IBD target cohorts, Cork (n = 87) and UCSD (n = 197), and test the gain on the three cohorts together. This gives 414 target samples. We compare TaxoMFM with the strongest baseline in each cohort. We perform paired resampling within each cohort and average the gain over the three cohorts, with 10,000 bootstrap resamples. Across the three cohorts, TaxoMFM achieves a mean AUROC of 0.765 (95% CI 0.719 to 0.811), with a mean gain of 0.041 over the strongest baseline in each cohort (two-sided P=8.7×10−3P = 8.7 \times 10^{-3}).

We run the same paired bootstrap on the other two target cohorts in Table 5. For colorectal cancer (n = 194), the gain over BiomeGPT is 0.075 (two-sided P=2.6×10−3P = 2.6 \times 10^{-3}). For type 2 diabetes (n = 783), the gain over the Species-only variant is 0.039 (two-sided P=6.7×10−3P = 6.7 \times 10^{-3}).

The table below reports bootstrap 95% confidence intervals for Table 5. We will add these results to the revised Table 5.

ModelCRC (n = 194)T2D (n = 783)IBD (n = 130)
Random Forest0.619 [0.537–0.698]0.634 [0.573–0.694]0.631 [0.531–0.737]
MIOSTONE0.643 [0.561–0.720]0.562 [0.513–0.615]0.693 [0.601–0.774]
PhyloGCNE0.610 [0.531–0.693]0.602 [0.548–0.658]0.567 [0.468–0.665]
MGM0.744 [0.672–0.813]0.670 [0.618–0.719]0.664 [0.565–0.763]
BiomeGPT0.763 [0.686–0.831]0.683 [0.629–0.731]0.679 [0.594–0.758]
Species-only0.759 [0.691–0.821]0.697 [0.651–0.739]0.688 [0.601–0.766]
TaxoMFM0.838 [0.778–0.893]0.736 [0.697–0.775]0.722 [0.638–0.796]
AuthorsOpenReview ↗

Response to Reviewer (Part 3/3)

  1. Q3: In Table 8, TaxoMFM retains 81.44% of the abundance while BiomeGPT retains only 45.79%. How much of the improved vocabulary robustness is attributable to ancestor routing rather than the learned representations?

Thank you for this question. The cross-vocabulary robustness in Table 8 has two parts: how much abundance is retained, and whether the same sample can be matched. The retention gap you mention (81.44% vs. 45.79%) comes entirely from ancestor routing. Routing decides which abundance is retained before the encoder reads the data, so it is independent of the learned representation. Sample matching depends on both. To separate them, we add two controls that need no pretraining.

Ancestor routing works as follows. The same sample is expressed in the current and the legacy vocabulary. If a species has no match in the legacy vocabulary, its abundance is assigned to its genus. If the genus has no match, it goes to the family, and then to the order. This is the nearest rank that both vocabularies recognize. Without routing, this abundance is discarded.

We compare three matching methods: (1) direct comparison of abundances at the species, genus, family and order ranks, without an encoder; (2) a randomly initialized, untrained TaxoMFM encoder; (3) the pretrained TaxoMFM encoder, as in Table 8. For each method, we compute bidirectional R@1 (Appendix C.4) with routing off and on.

Matching methodRouting offRouting onGain from routing
Direct abundance comparison17.99%59.44%+41.45
Randomly initialized TaxoMFM encoder5.63%25.32%+19.69
Pretrained TaxoMFM encoder67.69%86.64%+18.95

Reading across a row shows the effect of routing. Reading down a column shows the effect of the representation. Routing raises R@1 for all three methods, so the retained abundance itself helps identify the same sample. With routing on, the pretrained encoder reaches 86.64%. This is 27.20 points above direct abundance comparison and 61.32 points above the randomly initialized encoder. The random encoder has the same architecture as the pretrained one, yet it scores below direct comparison. The gain therefore comes from the representation learned in pretraining, not from the architecture. We will add these controls to Appendix.

Reviewer_JL5JOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes TaxoMFM, a novel foundation model for gut microbiome data that explicitly incorporates taxonomic hierarchies. The model has a taxonomy-aware encoder that makes the model permutation-invariant to its input. The decoder is a hierarchical compositional reconstruction model that recreates the complete community profile over the full vocabulary at each taxonomic level and enforces the composition within individual lineages. In a comprehensive experimental analysis that included many standard baselines, taxonomy-aware models, general microbiome foundation models, and other gut foundation models, TaxoMFM consistently outperformed the baselines. An ablation study shows the benefit of each component of the model.

Critical strengths
  • Proposes a new foundation model for microbiome data, with an emphasis on the gut microbiome.
  • Incorporates several mechanisms justified by the characteristics of microbiome data: sparse, high-dimensional, compositional, and hierarchical.
  • Introduces a new gut microbiome dataset, TaxoGut-HPP, which comprises 231 gut microbiome studies totaling ~100k samples.
  • Provides a comprehensive comparison of downstream task performance against a large set of baselines.
  • Includes an ablation study and an analysis of the taxonomy-aware pretraining.
Critical weaknesses

The main weakness of the paper is the lack of important details about the model itself and pre-training/fine-tuning steps. I strongly encourage the authors to clarify and include the following information in the manuscript:

Model Description:

  1. In Equation 1, are the abundance profiles raw read counts, or are they a normalized quantity, like CLR?
  2. What's the model's embedding dimensionality? What are the number of trainable parameters in the model?
  3. What's the maximum sequence length? For microbiome samples with many "active" nodes, what happens if the sequence is longer than max_seq_len? Any truncation? if so, how is that performed?
  4. What's the masking ratio? How are the masked subtrees chosen?
  5. In Equation 8, how is alpha_r obtained? Is it learned?
  6. At inference time, are all nodes tagged as "observed", or is E_state dropped?

Experimental Analysis: 7) Were all the baseline models available, or did the authors (re-)implement any of them? 8) How was the TaxoMFM model fine-tuned for the downstream tasks? Just replaced the decoder by a randomly initialized classification head and trained it? 9) Any hyper-parameter tuning for the Elastic-Net and Random Forest models?

Questions for authors

This is a solid paper with many contributions and careful design of a transformer model that incorporates the characteristics of the microbiome data. I enjoyed reading it. I strongly encourage the authors to clarify the points raised in the "Critical Weaknesses" section. I can then adjust my score accordingly.

4 comments
AuthorsOpenReview ↗

Response to Reviewer (Part 1/2)

  1. W1: In Equation 1, are the abundance profiles raw read counts, or are they a normalized quantity, like CLR?

Thank you for this question. The input to Eq. (1) is the species-level relative abundance, which sums to one within each sample. It is not a raw read count. We sum relative abundances along the taxonomy to obtain the genus, family and order abundances. Relative abundances are additive, so each rank still sums to one within a sample. This matches x^root=1\hat{x}_{\mathrm{root}} = 1 in Eq. (7). We then apply a smoothed CLR transform within each rank:

x~v=log⁡(xv+ε)−1∣Vr∣∑u∈Vrlog⁡(xu+ε).\tilde x_v = \log(x_v + \varepsilon) - \frac{1}{|V_r|} \sum_{u \in V_r} \log(x_u + \varepsilon).

Here VrV_r is the set of active nodes at rank rr, and ε=10−5\varepsilon = 10^{-5}. The transformed value is the input to the MLP in Eq. (2). In the revised version, we will add this step between Eq. (1) and Eq. (2) and change MLP(xv)\mathrm{MLP}(x_v) to MLP(x~v)\mathrm{MLP}(\tilde{x}_v).

  1. W2: What's the model's embedding dimensionality? What are the number of trainable parameters in the model?

Thank you for raising this point. The model dimension is d=512d = 512, and the pretraining model has 19.04 M trainable parameters.

ComponentParametersRole
Encoder (Table A11)18.25 MRepresentation extraction
Hierarchical decoder0.79 MAbundance reconstruction, pretraining only
Pretraining model19.04 MEncoder + decoder
Two-layer MLP head0.264 MTrained on the frozen encoder for downstream tasks

EtaxonE_{\mathrm{taxon}}, ErankE_{\mathrm{rank}}, EstateE_{\mathrm{state}} and the MLP output in Eq. (2), and EedgeE_{\mathrm{edge}} in Eq. (6), are all 512-dimensional.

  1. W3: What's the maximum sequence length? For microbiome samples with many "active" nodes, what happens if the sequence is longer than max_seq_len? Any truncation? if so, how is that performed?

Thank you for this detailed question. We set the input limits from the sequence lengths observed in our data, so that every active node of every sample enters the encoder. The largest sample has 322 detected species. With the genus, family and order nodes, the longest sequence has 707 tokens. We therefore set the limits to 512 species nodes and 768 tokens in total. No sample is truncated. Each sample keeps all its detected species and their ancestor nodes.

  1. W4: What's the masking ratio? How are the masked subtrees chosen?

Thank you for your valuable question. Each time a sample is loaded, we choose one rank at random between family and genus, among the ranks that leave visible species after masking. From the parents with non-zero abundance at this rank, we draw about 10% uniformly without replacement, and at least one. Each selected parent is the root of a subtree. We mask the abundance of this parent and of all its descendants. If neither rank leaves visible species, we skip subtree masking for this sample in this pass.

  1. W5: In Equation 8, how is alpha_r obtained? Is it learned?

Thank you for pointing this out. αr\alpha_r is a fixed weight and is not learned. We compute the reconstruction mean squared error in smoothed CLR space at each of the four ranks (species, genus, family, order). We then take the arithmetic mean of the four losses, so αr=1/4\alpha_r = 1/4 for every rank. We will state this value next to Eq. (8) in the revised version.

  1. W6: At inference time, are all nodes tagged as "observed", or is E_state dropped?

Thank you for this careful question. At downstream inference, all nodes are tagged as observed, and the state embedding term in Eq. (2) is kept. The state embedding distinguishes visible from masked nodes during masked pretraining. With no masking at inference, every node receives the same vector Estate[observed]E_{\mathrm{state}}[\mathrm{observed}]. We will state this after Eq. (2) in the revised version.

  1. W7: Were all the baseline models available, or did the authors (re-)implement any of them?

Thank you for asking about this. We implemented, trained and evaluated all baseline models ourselves. For models with public code, we built on the authors' released code. The only exception is BiomeGPT. We did not find public code for it, so we implemented it from the architecture and training procedure described in the original paper. For a fair comparison, all foundation models were pretrained on the same TaxoGut-HPP data. On each downstream task, all models use the same data splits and evaluation metrics. We will describe the implementation of each baseline in Section 4.1 of the revised version.

AuthorsOpenReview ↗

Response to Reviewer (Part 2/2)

  1. W8: How was the TaxoMFM model fine-tuned for the downstream tasks? Just replaced the decoder by a randomly initialized classification head and trained it?

Thank you for this thoughtful question. After pretraining, we remove the hierarchical decoder and freeze the encoder. The encoder extracts a 512-dimensional representation for each sample. For each downstream task, this representation is fed to a randomly initialized two-layer MLP head. Only the head is trained, using the training data of that task. The same pretrained encoder is therefore reused across all downstream tasks. We will describe this procedure in the revised version.

  1. W9: Any hyper-parameter tuning for the Elastic-Net and Random Forest models?

Thank you for this question. Elastic Net and Random Forest use fixed, preset hyperparameters. We did not run a hyperparameter search for either model.

  1. Q1: This is a solid paper with many contributions and careful design of a transformer model that incorporates the characteristics of the microbiome data. I enjoyed reading it. I strongly encourage the authors to clarify the points raised in the "Critical Weaknesses" section. I can then adjust my score accordingly.

Thank you for your positive assessment of our work! We have answered each point from the "Critical Weaknesses" section in our replies to W1–W9, and we will incorporate all of these clarifications into the revised manuscript. Your comments guided us to describe the model design and the downstream task setup more clearly, and they have improved the quality of the paper. We are happy to discuss any point that needs further clarification.

Reviewer_JL5JOpenReview ↗

Missing information are now present

Thank you for the detailed responses to my comments. Based on your clarifications, I am raising my score accordingly. This increase is conditional on the clarifications in your response being incorporated into the final manuscript. As this venue does not allow revised submissions, I am relying on you to make these changes, and I ask that you ensure they appear in the camera-ready version.

Thank you for your careful review and constructive feedback, which have helped us improve the clarity and quality of our paper. We will ensure that all clarifications provided in our responses to W1–W9 are incorporated into the camera-ready manuscript.

Reviewer_DkhTOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors have proposed a foundation model for human gut microbiome. In particular, taxonomic structure is proposed to be encoded inside the foundation model.

Critical strengths
  1. Taxonomy structure is important and has been proposed into foundation modelling.
  2. The foundation model has been comprehensively trained on 231 source cohorts.
  3. Source code is opened on GitHub for scientific reproducibility.
  4. The foundation model has been comprehensively compared with other models in Tables 2 and
Critical weaknesses
  1. There are many parameter settings which are not clear from writing.
  2. The results are not very impressive and I even question its statistical significance in p-value.
Questions for authors
  1. How to set the weight in equation (9) ?
  2. How to balance the recovery loss and hierarchical loss in lambdas in equation (10) ?
4 comments
AuthorsOpenReview ↗

Response to Reviewer (Part 1/2)

  1. W1: There are many parameter settings which are not clear from writing.

Thank you for this comment. The settings are listed below. We will collect them in one appendix table in the camera-ready version.

  • Architecture. 8 encoder layers, 8 attention heads per layer, model dimension 512, feed-forward dimension 1024, dropout 0.1. The encoder has 18.25 M parameters (Table A11).
  • Optimization. AdamW, learning rate 10−410^{-4}, weight decay 0.01, BF16 mixed precision.
  • Eq. (8). We add 10−510^{-5} to each relative abundance before the smoothed CLR transform. We average the four rank-level reconstruction losses, so αr=1/4\alpha_r = 1/4 for species, genus, family and order. This value is fixed and was not tuned.
  • Eq. (9) and Eq. (10). Our replies to Q1 and Q2 give the parent weights and the loss coefficients.
  1. W2: The results are not very impressive and I even question its statistical significance in p-value.

Thank you for raising this point. We address the size of the gains and their statistical significance below.

Size of the gains. The gains are small where the baselines are already near the ceiling. Among the 26 public disease tasks, the five smallest margins are: soil-transmitted helminths +0.001, Behçet's disease +0.003, acute diarrhea +0.003, atherosclerotic cardiovascular disease +0.007 and Graves' disease +0.007. The best baseline on these tasks reaches an AUROC of 0.986–0.993. As a share of the room the best baseline leaves (1 − its AUROC), these gains are 12.5%–58.3%, with a median of 42.9%. For example, on atherosclerotic cardiovascular disease, the best baseline scores 0.988 and leaves 0.012; TaxoMFM gains 0.007 of it, or 58.3%. The gains are larger on harder tasks: +0.046 on GI sensitivity, +0.036 on gastro-oesophageal reflux disease, +0.035 on current abdominal pain and +0.075 on zero-shot colorectal cancer transfer. The relative gain in R2R^2 is 31.2% on velocity and 30.6% on BMI.

Statistical significance. Four of the five near-ceiling tasks have very few positive cases: 14 for soil-transmitted helminths, 20 for Behçet's disease, 39 for acute diarrhea and 100 for Graves' disease, against 15,796 negatives. With so few positive cases, a statistical test on a single task is not informative. The fifth task, atherosclerotic cardiovascular disease, has 580 positive cases. On this task, DeLong's test for TaxoMFM against BiomeGPT gives a two-sided P value of 5.36 × 10⁻⁵.

We also add paired tests on the nine representative tasks from our ablation study (Table 4). On each task, we compare TaxoMFM with the second-best model. We use DeLong's test for the binary classification tasks and a paired bootstrap with 10,000 resamples for the regression tasks. The two-sided P values are 4.18×10−74.18 \times 10^{-7} for colorectal cancer, 9.91×10−69.91 \times 10^{-6} for IBD, 1.24×10−41.24 \times 10^{-4} for type 2 diabetes, 2.34×10−92.34 \times 10^{-9} for current abdominal pain, 1.19×10−51.19 \times 10^{-5} for current bloating, 3.32×10−23.32 \times 10^{-2} for irritable bowel syndrome, 1.8×10−31.8 \times 10^{-3} for the attenuation coefficient, 1.0×10−31.0 \times 10^{-3} for speed of sound and 3.1×10−33.1 \times 10^{-3} for velocity. After Holm correction for multiple comparisons, all nine tasks remain at P<0.05P < 0.05.

Consistency across tasks. TaxoMFM ranks first on 46 of the 48 tasks reported in the paper. A two-sided sign test gives P=8.4×10−12P = 8.4 \times 10^{-12} over all 48 tasks and P=8.0×10−7P = 8.0 \times 10^{-7} over the 26 public disease tasks. We will add these results to the appendix of the revised version.

AuthorsOpenReview ↗

Response to Reviewer (Part 2/2)

  1. Q1: How to set the weight in equation (9) ?

Thank you for this question. ωp\omega_p is computed from the data. Eq. (9) has no free weight parameter.

For each sample and each parent pp with target abundance xp>0x_p > 0,

ωp=xp∑p′∈Prxp′.\omega_p = \frac{\sqrt{x_p}}{\sum_{p' \in P_r} \sqrt{x_{p'}}}.

Here PrP_r is the set of parents at the same rank rr as pp, and xpx_p comes from the uncorrupted target profile. The weights are normalized within each of the four parent ranks (root, order, family, genus). The four ranks therefore contribute equally to LhierL_{\mathrm{hier}}, as with the uniform αr\alpha_r in Eq. (8).

The square root compresses a dynamic range of about five orders of magnitude. We will state this weighting rule next to Eq. (9) in the revised version.

  1. Q2: How to balance the recovery loss and hierarchical loss in lambdas in equation (10) ?

Thank you for this question. We set λrec=λhier=1\lambda_{\mathrm{rec}} = \lambda_{\mathrm{hier}} = 1. The two terms supervise the same reconstruction at two granularities and have equal standing, so they enter the total loss with the same coefficient.

Before the final pretraining run, we checked the sensitivity to this ratio. We set λrec=1\lambda_{\mathrm{rec}} = 1 and varied λhier\lambda_{\mathrm{hier}} over 0.25, 0.5, 1 and 2. All other settings were unchanged. Each model was pretrained separately and evaluated on all downstream tasks with the same splits.

λrec:λhier\lambda_{\mathrm{rec}} : \lambda_{\mathrm{hier}}Mean AUROCMean Spearman ρ\rhoMean R2R^2
1 : 0.250.9010.3340.141
1 : 0.50.9070.3400.144
1 : 10.9110.3430.147
1 : 20.9050.3380.139

We selected the 1:1 ratio because it achieved the highest mean AUROC, Spearman's ρ\rho and R2R^2 within this range.

AuthorsOpenReview ↗

Followup on Rebuttal

Dear Reviewer DkhT,

We thank the reviewer for the careful and precise review. We clarified how the weights in Eq. (9) are computed, showed that the 1:1 loss ratio in Eq. (10) performed best among the settings tested, and reported paired significance tests with Holm correction across nine tasks. As the review period draws to a close, could you please let us know whether our response addresses your concerns or if any point needs further clarification?

Regards Authors

Thank you for the replies. I have raised the score to maximum.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.