acceptodds
Under review as a conference paper at ICLR 2027

BIOTRAM: Biochemistry-informed Transformer for Relative Abundance via Metabarcoding

Abstract

Arthropod monitoring at scale begins with metabarcoding of bulk samples: trap collections of hundreds to thousands of specimens processed as one unit, extraction of DNA from the bulked specimens, then sequencing of the DNA from all specimens together. Metabarcoding reveals which taxa such a sample contains, but due to factors such as differing DNA amplification rates and body sizes, its read counts do not neatly correspond to how abundant the different taxa are relative to one another. This lack of relative abundance limits the data’s utility for the multitude of ecological analyses that depend on it. We propose BIOTRAM: a neural network that estimates per-taxon specimen counts by dividing the observed read counts by a learned read yield per specimen, which it factors into the DNA each specimen contributes and how efficiently that DNA is amplified. On a paired dataset of 1,188 bottles (800k specimens) processed both by metabarcoding and by comprehensive individual barcoding for ground-truthing, BIOTRAM reduces the Jensen–Shannon divergence between the estimated and the true composition from 0.3586 for raw read proportions (the usual proxy for abundance) to 0.1200. Trained on the same objective and inputs as four generic deep architectures, all with tuned loss weights, it reaches a count error 3.2% below that of the strongest of them and the lowest composition divergence. It also generalizes to out-of-domain data better, beating all baselines on bottles from unseen regions, later collection dates and unseen sequencing batches, and transferring to data from an unseen laboratory. Structuring the mapping from reads to counts around the steps of the experimental pipeline thus yields more accurate abundance estimates than leaving that mapping unconstrained.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Metabarcoding, Community abundance, DNA barcode, Transformer, Biodiversity

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_9yubOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose to estimate per-taxon relative abundance from DNA metabarcoding of bulk arthropod samples with a neural network built around the steps of the lab process. It then reverses this relationship by dividing the observed reads by a learned read yield per specimen. Two frozen external estimators supply the per-BIN inputs: (1) a body-size estimator built from segmenting 640k BOLD specimen images and fine-tuning BarcodeBERT to predict size from the barcode, and (2) a primer binding-efficiency estimator that scores primer sites retrieved from full-length barcodes and mitogenomes, with ridge-regression imputation for the rest. Separate heads predict body mass and presence from BarcodeMamba embeddings, taxonomy and bottle metadata. A self-attention module over all BINs in a bottle predicts the amplification term, and a small learned network merges the three PCR replicates. The model is trained with a negative-binomial likelihood plus a Jensen–Shannon divergence (JSD) composition loss, a presence BCE loss and a reads-per-specimen loss. The authors also contribute a paired dataset processed both by metabarcoding and by individual barcoding. The framework is evaluated on in-domain random splits, on spatial, temporal and sequencing-batch shifts, and on a small external dataset from another lab.

Critical strengths
  • S1: The dataset is very valuable, which is more than 10x larger than previous existing datasets.
  • S2: Model is derived from the lab process as a clear example of domain knowledge shaping the inductive bias.
  • S3: Each component is motivated by data analysis.
  • S4: Careful, fair tuning of baselines.
  • S5: Broad evaluation under distribution shift.
  • S6: The 30-block ablation, paired seed-for-seed with a noise band, shows that data and chemistry matter most.
Critical weaknesses
  • W1: Small gains over generic deep baselines.
  • W2: Potential circular ground truth for the bottles with excess specimens. In bottles where abundant morphospecies were counted by eye, those counts are assigned to BINs using the predictions of a model trained on the other bottles.
  • W3: Weak inputs for primer efficiency. The reverse primer efficiency is imputed for 98.8% of BINs, from a predictor with R2R^2 = 0.16 whose outputs are heavily shrunk toward the mean. The forward imputer is worse than a taxonomic mean.
  • W4: Statistical testing ignores variation between test bottles.
Questions for authors
  • Q1: Can you report paired bootstrap confidence intervals over test bottles (not only over seeds) for the JSD and MAE differences between BIOTRAM and the best baseline in Table 1?
  • Q2: How were the development weights (8, 0.15, 0.05) chosen, and on which model? If they were tuned on BIOTRAM, how should Table 3 be read as an "architecture-only" comparison?
  • Q3: Please reconcile the presence AUC values (0.9136 in Table 1 vs. 0.9213 in Appendix O).
  • Q4: How does performance change if BINs found by individual barcoding but missed by metabarcoding are kept in the evaluation (predicted as zero)? How many bottles hit the 768-BIN truncation, and what share of individuals does truncation remove?
2 comments
AuthorsOpenReview ↗

Reviewer 9yub individual reply: gains over baselines, incomplete ground truths and primer-efficiency input

Thank you for reading and giving us feedback on the paper. We respond to each of your questions and weaknesses that you identified below. Additionally, please see our general response which describes changes in the evaluation protocol, affecting all tables, and answers in full the concerns that several reviewers raised.

W1: Small gains over generic deep baselines.

We agree that the gap between the BIOTRAM architecture and the other deep baselines is only modest; point 4 of our general response explains what this comparison measures and how it relates to the larger gain over existing methods.

W2: Circular ground truth for the excess bottles.

The text about excess that raised the concern about circularity is wrong and we apologize for it: no label is completed by a model. Point 6 of our general response gives the details.

W3: Weak primer-efficiency inputs.

Your point is right and we, in fact, struggled to estimate the primer binding efficiency for the metabarcodes. The consensus sequences that we get after amplification are unfortunately a bad predictor for the binding sequences, outside of its range and we didn't have enough sequences that cover the primer binding sites (mitogenomes and full-length barcodes) to get a good coverage of the taxonomy, in particular for the reverse site efficiency. This is a current limitation of the model and we hope to obtain longer sequences in the future to mitigate it. Regarding your comment on the taxonomic mean, while it does indeed perform better than our imputer, it relies on the BIN’s full taxonomy at genus level or below, whereas our imputer needs only the amplicon. This matters because 77% of BINs in our metabarcoding dataset have no known genus name, so we cannot use it on the majority of the data. The data we fit the reverse primer efficiency model on has full taxonomic data, so we were able to measure the performance of the taxonomic mean as a baseline. To better show the effect of incomplete taxonomic labels, we have expanded this analysis to also show the performance of the mean for truncated taxonomy up to a specific rank (species, genus, family, order). With the taxonomy known to genus or species, the taxonomic mean predicts the forward efficiency slightly better than our barcode predictor (R2R^2 0.261 against 0.234); known only to family it is slightly worse (0.229), and to order it fails (0.043). For the reverse efficiency our predictor is better at every rank (0.162 against 0.095 at most). With each test BIN’s taxonomy cut to the ranks actually known for our BINs, the taxonomic mean reaches R2R^2 0.229 (forward) and 0.087 (reverse), against 0.234 and 0.162 for our predictor. We also retrained BIOTRAM with the taxonomic mean in place of every predicted efficiency: its JSD does not change (−0.0001, 95% CI −0.0010 to +0.0008). We have also added discussion of this to our Limitations section, where we state explicitly that, while primer binding efficiency belongs in the model’s structure, current reference data cannot estimate it well; single-specimen barcodes that keep the primer sites, and more mitogenomes, would improve it.

Edit: Re-organized content.

AuthorsOpenReview ↗

Reviewer 9yub individual reply: test bottles variation, development loss weights, presence AUC, missed BINs and BIN truncation

W4 & Q1 : Testing ignores variation between test bottles.

We agree with your point: with only 89 test bottles, the submitted results could depend on which bottles were randomly allocated to the test set. The revision replaces this split with nested 5-fold cross-validation over all bottles with complete counts (point 1 of our general response); this goes beyond what you asked for and gives a better estimate of performance.

We have also computed the paired bootstrap CIs over test bottles: Over the 419 test bottles (10,000 resamples), BIOTRAM minus each baseline with the recorded total is, for JSD: −0.0016 (95% CI −0.0033 to +0.0000) against the Transformer, −0.0013 (−0.0029 to +0.0002) against the 1-D CNN, −0.0041 (−0.0063 to −0.0020) against the MLP and −0.0051 (−0.0080 to −0.0027) against DeepSets. For MAE per bottle the interval excludes zero against every baseline (Transformer −0.027, −0.044 to −0.013), and so it does for AUC (Transformer +0.0057, +0.0029 to +0.0087). The JSD lead over the Transformer and the CNN is therefore small and not confirmed by the bootstrap, while the MAE per bottle and AUC leads are. The appendix gives these intervals, together with a two-level bootstrap that also resamples the runs of each bottle.

Q2: How were the development weights (8, 0.15, 0.05) chosen?

In the revision, every model’s hyper-parameters, loss weights included, are tuned inside each cross validation fold on validation bottles only, so no reported comparison uses weights chosen during BIOTRAM’s development. In the submission, these weights were chosen on BIOTRAM's validation set, during its development, and not updated afterward. The submitted Table 3 used this one setting for every model, so that only the architecture differed, while the main comparison (the submitted Table 1) tuned each model’s three weights separately, with the same protocol and budget (App. O); none of the five models, BIOTRAM included, selected (8, 0.15, 0.05). The architecture-only comparison (Table 3) now gives every baseline BIOTRAM’s tuned weights of the same fold, so that only the architecture differs: BIOTRAM’s JSD is then 0.0017 below that of the strongest baseline, against 0.0014 when each model has its own weights (App. I).

Q3: Presence AUC, 0.9136 against 0.9213.

Both values are right but measure different things; point 8 of our general response explains them and how the revision reports them. We thank the reviewer for making us aware of this inconsistency.

Q4a: Missed BINs.

We added this analysis (new App. U); point 7 of our general response gives the results.

Q4b: 768-BIN truncation.

This was an error in the text from a preliminary version of the model. The version of the methodology used in the submitted version did not have a 768-BIN limit. Every model was trained and scored on every BIN of every bottle. We apologise for this mistake and have now corrected the text accordingly.

Edit: Re-organized content.

Reviewer_eYFvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper tackles a real problem in insect monitoring where metabarcoding usually tells us which insect groups are present, but not how many individual insects were in the sample. The authors propose a new model BIOTRAM that tries to correct this by modeling some of the biology behind the DNA process. Instead of simply learning "DNA reads --> insect count," it considers factors such as body size, primer efficiency, amplification effects, and whether a detected taxon is actually present.

The dataset itself is a great resource. With 1188 samples, around 800000 individuals, and more than 132,000 BINs, it is to my best knowledge the biggest resource of such kind. The model performs slightly better than other methods, but it shows some robustness to changes in location, etc.

Critical strengths

The paper has a lot going on. The authors put together a very valuable dataset, a biologically motivated model, a wide range of baselines, and many many ablations, experiments and results analysis. The paper is genuinely easy to read and understand, and I actually enjoyed reading it. It does not feel like another AI slop paper. The figures and tables are very informative and overall a provide an additional angle to understand the results.

The dataset itself could potentially be a standalone paper with a great contribution. I.e., providing metabarcoded data paired with individual specimen counts at this scale is awesome. I also like that the model is not just another generic NN applied to the problem. The authors make use of biological and experimental metadata, and the architecture is clearly the result of thinking about the actual process rather than only treating this as a generic prediction task. The authors support the model with a very extensive set of experiments, ablations, and analyses. Maybe too many.

Critical weaknesses

Overall, there is a lot of great work in the paper, but I am somewhat do not understand why the authors submitted it into ICLR. Given the domain-specific nature of the contribution and the length (36 pages), I believe some high-impact journal might be a better fit.

Anyway, my main concern is simple but potentially very harmful. Given the Eq. 2 and Fig. 2, it seems the total number of specimens in the bottle is an input to the model. However, in deployment setting the whole point is that we only have the metabarcoding data. So where would this total specimen count come from in deployment?

To me this is a dealbreaker because the total count is basically the sum of the numbers that the model is trying to predict. If the true total count is given to the model, is can potentially just leak this information. I think the authors need to either remove this input and rerun the experiments, or demonstrate an easy way of estimating the total specimen count at deployment and evaluate the model using that estimate instead of the ground-truth value. Without that, and despite I really like the article, I cannot accept it in its current form.

Questions for authors
  • Q1: How much would performance change if specimen count is not given to the model as an input? 


  • Q2: Given the improvement over the baseline is small, how stable is it if the test set itself changes, not just seed?


* Q3: How sure are you that the model’s internal estimates (e.g., body size), really represent biological effects and are not just some mathematical variables?


  • Q4: Given individual barcoding can miss some insects, how much could these missing insects affect the training labels and the model’s results?


  • Q5: For many BINs, some metadata has been predicted, not measured. How much does the final performance depend on those predictions being accurate/inacurate?


* Q6: BIOTRAM seems weaker on BINs it has never seen while training. How serious is this? In real-world, the model will always encounter something it has never seen before.

2 comments
AuthorsOpenReview ↗

Reviewer eYFv individual reply: venue choice, total specimen count, test set variation and latent & biology alignment

Thank you for your review, we appreciate your feedback and are glad you enjoyed the paper. We answer each of your questions and weaknesses that you identified below, you'll also find a general response that describes changes affecting all tables, and answers in full the concerns that several reviewers raised.

Choice of venue

First of all, this paper is a collaboration between machine learning scientists and biodiversity scientists (barcoding experts). We chose to submit it to ICLR because, as the reviewer points out, it is more than a dataset release. The scale of data finally allows us to train and evaluate state-of-the-art deep learning methods on this problem, which, if it works, makes metabarcoding much more valuable to the biodiversity community. As much of the paper is focused on our deep learning and biochemistry-hybrid method, we thought it would interest the readership and be a good venue to get this community interested in this prediction problem. We think BIOTRAM is just the start to methods development for predicting relative abundance from metabarcoding.

Main concern and Q1: the total specimen count.

You are right that the total specimen count was a required input to the model, and that a metabarcoding-only deployment does not necessarily have it. Point 2 of our general response answers this concern, including Q1: retraining BIOTRAM without the total raises its JSD by 0.0021 and its MAE per bottle by 0.069 (Table 9). We would value your view on whether this resolves the concern.

Q2: Stability when the test set changes.

The revision replaces the single test split with nested 5-fold cross-validation over all complete bottles (point 1 of our general response, which also gives how much BIOTRAM's JSD varies between and within test partitions); point 3 gives the paired tests against each baseline.

Q3: Do the internal estimates represent biology?

Partly; see point 5 of our general response: the product of the two terms, each BIN's reads per specimen, tracks its empirical value on held-out bottles (r = 0.684), but the split into body mass and amplification cannot yet be evaluated against data.

Edit: Re-organized content.

AuthorsOpenReview ↗

Reviewer eYFv individual reply: barcoding failures, input sensitivity, BIN never seen in training

Q4: Specimens missed by individual barcoding.

Individual barcoding does not sample: every specimen in a bottle is separated and sequenced. Sequencing failures do occur, though they affect labels for under 10% of specimens (Sec. 3). A failure lowers its BIN count by one; failure rates can differ between taxa (for example where the barcoding primers match a group poorly), so the labels under-count taxa that fail more often. A BIN where every specimen failed appears with reads but a zero count and is labelled absent. Failures are an inevitable part of barcoding.

Q5: Dependence on predicted metadata.

We ran a sensitivity analysis on the two per-BIN estimates, body size and primer efficiency, by adding noise to them at test time. Body size matters more than primer efficiency for every model: noise of one across-BIN standard deviation in the predicted body size raises BIOTRAM’s JSD by 0.0085, the least of the five models (CNN 0.0092, MLP 0.0105, Transformer 0.0113, DeepSets 0.0136), and the same noise in the primer efficiency raises it by 0.0011. Shuffling the predictions across BINs costs BIOTRAM 0.0161 JSD (body size) and 0.0022 (primer efficiency).

We can't, however, directly compare predicted and measured values at scale because measured values exist only for a few BINs (dry mass for 587 BINs; reverse primer sites for 1.2% of BINs), so most test rows have no measured equivalent. Where measured values exist, we compared them with the predictions on the same BINs (new App. V): measured values track a BIN's empirical log reads per specimen better than the held-out predictions, for the forward primer efficiency over 29,499 BINs (r = 0.22 against 0.18) and for body size over 509 weighed BINs (r = 0.36 against 0.29), so more accurate estimates should help.

Q6: BINs never seen in training.

They are 21% of test rows and differ in distribution, which the table doesn't show clearly: most are absent or single specimens, so their MAE is far lower (0.589 against 0.908 for seen rows in the revised evaluation, Table 6) and mostly measures how well zeros are predicted. To separate distribution from accuracy we added, for every model:

  • MAE as a function of the true count, one curve for seen and one for unseen BINs, with the mean and spread over runs
  • a comparison of seen and unseen rows after matching their distributions of true counts

Pooled over the runs, BIOTRAM’s MAE on unseen BINs is 0.612, against 0.908 on seen BINs. Reweighting the seen BINs on the unseen BINs true count distribution, we get a MAE of 0.529 so BIOTRAM is slightly less accurate on unseen BINs when their distribution is accounted for, same goes with every model (seen, matched: 0.544 to 0.578 for the baselines). A calibration curve also shows that BIOTRAM over predicts unseen BINs less than the baselines (log ratio of predicted to true count is +0.063 on average for BIOTRAM against +0.069 to +0.092 for the baselines). Unseen BINs that are present, however, are under-predicted by every model, and more than seen BINs with the same counts (App. W).

Edit: Re-organized content.

Reviewer_pcFPOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BIOTRAM, a biochemistry informed Transformer designed to estimate species abundances from metabarcoding read counts. The authors evaluate the method using 1,188 samples and report lower Jensen-Shannon divergence than existing approaches. Yet, the performance gains are modest relative to the complexity of the proposed architecture.

Critical strengths
  1. The paper introduces a valuable dataset comprising 1,188 samples and approximately 800k individual organisms.
  2. Incorporating biochemical processes into the model architecture is an interesting and potentially valuable direction.
  3. The authors conduct comparisons against multiple baselines, ablation studies, and evaluations under distribution shifts.
Critical weaknesses
  1. The improvement over the strongest general-purpose deep learning baseline is modest (from 0.1234 to 0.1200). This gain does not appear sufficient to justify the additional architectural complexity.
  2. Biomass and amplification factors are not independently identifiable. This raises concerns about whether the corresponding latent variables can be assigned meaningful physical interpretations. The paper's central interpretability claim remains insufficiently supported.
  3. BIOTRAM achieves a higher JSD than the existing GLMM baseline on data from an external laboratory. This result weakens the claim of superior generalization across experimental settings.
  4. BIOTRAM does not achieve the best results in presence AUC or Spearman correlation. Its advantage over TabICLv2 in count accuracy is also unclear. These results make it difficult to establish consistent superiority over existing methods.
  5. The ground-truth counts for 603 samples are incomplete and are corrected using a model-based procedure before training. This process may introduce systematic biases into the training data. The potential impact of these biases is not sufficiently investigated.
  6. BIOTRAM cannot predict species that are not detected through metabarcoding. Also, its robustness to changes in primers and experimental protocols has not been adequately demonstrated. These limitations raise concerns about its applicability in real-world settings.
Questions for authors
  1. What fundamental advantages does the proposed biochemistry informed architecture offer over simpler models?
  2. How do the authors justify the additional complexity given this relatively small gain?
  3. How can the physical meaning of the latent variables be established when biomass and amplification factors are not independently identifiable?
  4. What evidence supports interpreting these variables as distinct biochemical quantities?
  5. What motivates the use of BIOTRAM in practical deployment scenarios when it underperforms GLMM on external laboratory data?
2 comments
AuthorsOpenReview ↗

Reviewer pcFP individual reply: model complexity & benefits, latents interpretability, deployment on different protocol

Thank you for reading and giving us feedback on the paper. We answer each of your questions and weaknesses that you identified below. You'll also find a general response that describes changes affecting all tables, and answers in full the concerns that several reviewers raised.

W1 / Q1 / Q2: What does the structure buy, and does it justify its complexity?

  • Advantage over simpler models. The gain over existing methods is large, while the gap to our own deep baselines, which also share BIOTRAM's components, is modest; point 4 of our general response explains this comparison and our contributions.
  • Its complexity. The structure we introduced doesn't add size: every deep model, BIOTRAM included, has 10M parameters (±2%), since we control for the parameter count. Its complexity is the fixed arithmetic of Eq. 2 around three small networks. In return, the structure gives: its intermediate output, each BIN's reads per specimen that can be checked against data (r = 0.684 on held-out bottles) and the presence probability for each BIN. The deep baselines on the other hand give only the final counts.
  • What we improved to mitigate this issue. To make the comparison with the deep baselines as robust as possible, we moved to nested cross-validation (point 1 of our general response), under which BIOTRAM has the best mean of the five deep models on all three main metrics (point 3).

W2 / Q3 / Q4: Can the latent terms be read as distinct biochemical quantities?

We agree with your concern; point 5 of our general response answers it and describes how we revised the claims.

W3 / Q5: The GLMM on the external dataset.

The external Collaterals dataset is extremely different to our training data, and a stress-test for out-of-domain generalization in this setting. The external bottles come from a different country to all our training data; have one PCR replicate instead of three; have no location, date, or biomass records to input to the model; and the extraction method differs from ours. The test shows transfer in an extreme setting, but it is not the intended deployment. We have adjusted the text to make these differences clearer.

The GLMM does not degrade on the Collaterals dataset (JSD 0.193 and 0.188, against 0.2052 in domain) because it uses only reads and taxonomy; lacking the capacity to input most of these features omitted or changed with the change in dataset, it is not sensitive to them being missing or changed. BIOTRAM on the other hand, has a harder time to adapt to the metabarcoding protocol having far more read counts sequenced than any training bottle (App. M).

BIOTRAM was trained on Malaise trap bottles from the LIFEPLAN project and the Global Malaise Program. We highly recommend applying this trained model to bottles collected under these programs or using similar protocols. To deploy the best model in highly different deployment conditions, it would be ideal to use the method we propose to train a new model (or fine-tune our pretrained one), provided sufficient data exists to do so. This will bake in the correct priors to have a performance gain in the deployment setting.

We rephrased the external-data paragraph accordingly, clarifying it is far-OOD data, whereas the other partitions we explore in Table 2 are near-OOD settings. We would also be glad to evaluate the models on any other paired metabarcoding dataset you know of.

Edit: Re-organized content.

AuthorsOpenReview ↗

Reviewer pcFP individual reply: performance comparisons, incomplete ground truth and undetected taxa

W4: Presence AUC, Spearman and TabICLv2.

You are right: on AUC BIOTRAM is behind the four tuned baselines, and on Spearman correlation it is level with three. We did not claim superiority on these ranking metrics. That said, in the revised evaluation (nested 5-fold cross-validation), BIOTRAM’s AUC is the best of the five deep models (points 3 and 8 of our general response), ahead of all four in a paired test over the runs. More generally, we report a wide range of metrics with the intention of being transparent with our results and the performance of the models, and to have all the information to discuss the benefits and differences when leveraging biochemistry knowledge in a model design. Against TabICLv2, BIOTRAM is ahead on JSD (0.1383 against 0.1428) but behind on MAE per bottle (0.729 against 0.689), AUC and Spearman. But the primary advantage over TabICLv2 is inference cost (see Figure 12c). Using exclusively in-context learning and no training on our data means the whole training set is processed for every inference step, requiring 250,000 times more compute per bottle for TabICLv2 than BIOTRAM. The revision now states these trade-offs and the evaluation with cross-validation also makes all metrics and comparisons more robust.

W5: Incomplete ground truth of the excess bottles.

This was an error in our text, and we apologise for it: no label is completed by a model. Point 6 of our general response explains how these bottles are used in training and what they bring.

W6: Undetected taxa and other protocols.

You are right that BIOTRAM cannot predict the presence of species not detected by metabarcoding. We have clarified this in the Discussion, but do not think it should be held strongly against us since such a feat is impossible. An estimator built on metabarcoding can only report taxa that leave reads; taxa lost at DNA extraction or amplification are invisible to it, as to any read-based method, including the GLMM baseline. Undetected taxa are therefore a limitation of the metabarcoding process rather than of the models themselves, and addressing it is far beyond the scope of this work. Point 7 of our general response quantifies these losses.

Edit: Re-organized content.

Reviewer_12TiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses estimating per-species (BIN) specimen abundance in a Malaise-trap bottle from metabarcoding read counts, which are not linearly related to specimen counts. It contributes (1) a paired dataset of 1,188 bottles (about 800k specimens, 578 sites) processed with both metabarcoding and individual barcoding, and (2) BIOTRAM, a transformer-based model built on a forward model in which reads depend on specimen count, body mass, and PCR amplification (primer binding). The model estimates the latter two terms and inverts the relationship to obtain counts, which it describes as physically interpretable.

Validation includes random splits, spatial, temporal, and sequencing-lane shifts, and a small external set of 15 bottles, against learned baselines and a hurdle mixed model, with ecological metrics (Shannon diversity, Bray–Curtis). BIOTRAM reaches a JSD of 0.1200 (0.3586 for raw read proportions; 0.1234 for the best baseline) and a 3.2% lower MAE, trails several baselines on presence AUC, and all models degrade under shift.

Critical strengths
  1. A large, well-characterized paired dataset. The paper assembles 1,188 bottles from 578 sites (2014–2025) with both metabarcoding and individual barcoding ground truth, more than an order of magnitude larger than previous paired data. Such data are expensive to produce and are the main bottleneck for learning-based abundance estimation, so this resource (with code, checkpoints and data promised upon acceptance) is likely to be valuable to the community.

2 Problem formulation and evaluation are both grounded in ecological practice The paper clearly identifies the real bottleneck of metabarcoding: it is cheap and scalable, but read counts are not linearly related to specimen abundance, which is important for ecological monitoring. The task is therefore posed at the unit ecologists work with (a bottle, with per-BIN specimen counts as output), and the evaluation matches it: spatial, temporal, and sequencing-lane shifts, an external dataset from another laboratory, and ecological metrics (Shannon diversity, Bray–Curtis Mantel correlation).

Critical weaknesses
  1. The role of the bottle total ν_b is unclear for deployment. If νbν_b is the sum of the true specimen counts in the bottle (based on Line 278, νbν_b the total specimen count"), then the model receives the bottle total as an input (Eq. 2) and gives the model the overall count scale. The paper does not state how νbν_b would be obtained in practice, where counting specimens is what metabarcoding aims to avoid, and the Limitations section doesn't mention it. In addition, based on Table 9 (ablation results), removing it changes count-space relative error the most (+0.0596), though JSD changes little (+0.003). It is therefore unclear how well the count estimates hold when ν_b is unknown or only estimated. This matters for users who actually want to use this tool.

  2. The interpretability claim might need more evidence. The paper presents the body-mass term mm and the log-amplification term aa in Eq. (1) as physically interpretable. However, the counts seem to identify only their product meame^{a}, and the amplification module also sees the BIN's own read, so the likelihood alone pins down neither term. The independent checks appear to support body size (r=0.86r = 0.86-0.880.88, Line 441) more than primer efficiency (r=0.21r = 0.21, Line 442), and the remaining check is on the product only (r=0.627r = 0.627; 0.4610.461 for the read-blind variant). If so, mm and aa may be better viewed as useful correction terms than as separately identified physical quantities. This matters because interpretability is the main property claimed beyond generic baselines.

Questions for authors
  1. How is νb\nu_b obtained at test time: is it the sum of the ground-truth counts in the bottle? Could the authors report how the count-space error changes when νb\nu_b is unknown or only estimated (e.g., noisy or read-based estimates)?

  2. Could the authors show whether the two terms are separately recoverable, for example on simulated data where the true aa and mm are known, or on the weighed BINs?

  3. The read-blind variant is described as the cleaner one for interpreting the latent terms. Could the authors report its results on an external benchmarks, so that the accuracy cost of interpretability is clear, and say which variant they recommend for interpretation?

Minor points

  1. MAE is used in Figure 2 but only defined later. Please define it at first use.
  2. The order of Phase 1a/1b in Figure 2 (primer binding, then body size) differs from Sec. 4.1 (body size, then primer binding). Using the same order would make it easier to read.
  3. Table 1 and Table 8 appear to report the same retrained BIOTRAM model (JSD and MAE are the same), but the presence AUC differs (0.9136 vs. 0.9213). Could the authors clarify which value is correct?
  4. The Results section reports 864 training bottles (Line 361-363), but 409 + 603 = 1,012 (Appx. E, Line 1077). Could the authors clarify whether some bottles were excluded?
1 comment
AuthorsOpenReview ↗

Reviewer 12Ti individual reply

Thank you for reading and giving us feedback on the paper. We respond to each of your questions and weaknesses that you identified below. Additionally, please see our general response which describes changes in the evaluation protocol, affecting all tables, and answers in full the concerns that several reviewers raised.

W1 / Q1: How is νb\nu_b obtained at test time, and how do the counts hold without it?

νb\nu_b is indeed the sum of the individually barcoded counts, at training and at test time. Point 2 of our general response answers both questions, including the two analyses you suggested: the model's MAE with a noisy total, and with a total estimated from the reads, the number of BINs and the wet biomass.

W2 / Q2: Are the two terms separately recoverable?

We agree with your reading; point 5 of our general response gives our answer and the revised claims. Thank you for suggesting the simulation study, which we added to the Discussion as future work.

Q3: The read-blind variant on external data.

We added it to the external-data table (App. M) as suggested. On the external bottles, the read-blind variant reaches a JSD of 0.216 (lysis) and 0.209 (homogenate), against 0.224 and 0.197 for the main model, with a much lower MAE per bottle (2.58 and 2.37, against 4.65 and 3.05). In domain (App. I), it costs 0.0050 JSD and 0.064 MAE per bottle relative to the main model, which places it level with two of the four deep baselines, on both JSD and MAE per bottle. For interpreting the latent terms we recommend the read-blind variant, because its amplification term cannot reproduce a BIN's own read. For accuracy, we recommend the main model. The paper now says so explicitly, with numbers to support the argument.

Minor points.

1 and 2: We have fixed both points in the updated submission.

  1. AUC values. Both values come from the same runs but measure different things; we clarified what they correspond to in point 8 of our general response and made them consistent through the paper.

  2. Number of bottles. 864 = 409 excess-free training bottles + 455 of the 603 bottles with excess. The other 148 were excluded because excess specimens affect more than 5% of their BINs. The revision states the new counts (Sec. 5 and Appendices E and S). They change in the revision since the evaluation moves to cross-validation (see point 1 of our general response): now each of the 419 complete bottles is a test bottle in four of the 20 runs, and each run trains on three of the five partitions of complete bottles (about 251 bottles) plus four fifths of the 769 incomplete bottles (about 615), 867 training bottles per run on average; the remaining fifth of the incomplete bottles is held out to measure what training on them brings (App. E).

Edit: Re-organized content.

Comments on the paper

5 comments
AuthorsOpenReview ↗

General reply: revised evaluation protocol

We thank the reviewers for their constructive reviews and feedback. In this post we provide an overview of broad changes to the experimental protocol, revised in response to reviewer feedback. We also answer here, once, the concerns raised by more than one reviewer; our replies to each reviewer point to the relevant section below.

1. Stability under changing test set

Reviewers eYFv and 9yub asked about the stability of our results under changes to the test set. In order to address this, we have modified our evaluation protocol to use nested cross-validation instead of a fixed split. This improved methodology provides a better estimation of the performance of the methods involved in the study. But as a consequence, all results values reported in the tables of the paper have changed. Nonetheless, the main conclusions of our work remain unchanged. To ensure all reviewers are aware of the change, we describe it here.

The submission tested every model on a single split of 89 bottles and compared models over different initialization seeds. Those tests ignore variation between test bottles, and the result rests on one test set. The revision uses nested 5-fold cross-validation over all 419 bottles with complete counts, so each of these bottles serves as a test bottle. The bottles where one or more taxa in apparent excess were withheld from barcoding (and therefore do not have a complete count) are still excluded from the main evaluation. The new protocol includes the following:

  • Each model's hyper-parameters, loss weights included, are selected inside each fold, on validation bottles only.
  • The per-BIN estimators of "Phase 1a/b" (body size and primer efficiency) are also fit for each fold, on that fold's training bottles only.
  • Each test partition is predicted by 4 models, one per validation partition: the selected configuration is retrained once per validation partition, on the other three partitions. This gives 20 runs per model; their mean equals the mean of the 5 test-partition means, and we report it ± one standard error computed from those 5 means.
  • The new App. S shows how much the spread comes from the test partitions and how much from the training variability (initialisation noise, validation set, etc): for BIOTRAM's test JSD, the standard deviation is 0.0060 between the five test partitions and 0.0022 between the runs within a partition.

Edit: Re-organized content.

Thank you for the detailed clarifications and revisions. While the revised evaluation and methodological clarifications address some of my concerns, my main concerns regarding the modest benefits of the architecture over simpler deep learning baselines, the lack of independent validation of the latent variables, and generalization remain. I am also concerned that the code and dataset are not currently available for verification, particularly given the relatively small performance differences. I therefore maintain my original rating.

AuthorsOpenReview ↗

General reply 2: total specimen count and new results

2. Availability of total specimen count

Reviewers 12Ti and eYFv were concerned that the bottle’s total specimen count νb\nu_b had to be provided as an input to the model, which users may not know.

  • What νb\nu_b is. It is the sum of the individually barcoded counts over the bottle's BINs (plus, for an incomplete bottle, i.e. one where specimens of very abundant taxa were counted but not barcoded, these withheld specimens), at training and at test time. The four deep baselines received the same value, so the comparison across architectures is matched on it.
  • What information it carries. It sets the scale of the counts, not their split between BINs. Read counts alone only contain enough information to estimate the specimen distribution in the sample, because the number of reads a bottle gets is set by the sequencing protocol (only a roughly fixed number of amplified DNA strands are sequenced), not by how many specimens it holds. This is why removing νb\nu_b barely moves JSD but strongly moves MAE: absolute counts need an external scale. In the revised Table 9, retraining BIOTRAM without it costs +0.0021 JSD and +0.069 MAE per bottle, the second-largest effect on count after the cross-BIN attention.
  • Obtaining it without barcoding. A user can get this number by manually counting the specimens in their bottle, or at least inspecting it closely enough to estimate the count. While this is labour-intensive, it takes substantially less time than individual barcoding and doesn’t require specialized training, consumables, or access to a barcoding protocol. A new sensitivity analysis shows that the model tolerates a reasonable approximation: a random error of about 25% in the count raises BIOTRAM’s JSD by 0.0001 and its MAE per bottle by 0.006 on average, and an error of about 50% by 0.0003 and 0.023 respectively; even at 50% error, its MAE per bottle (0.752) stays below the Transformer’s with the same error (0.760).
  • Making it optional. We trained an imputation model to estimate this input from other features available (bottles’ read count, number of unique BINs and total wet biomass); it recovers the log total with R2=0.717R^2 = 0.717. Every model is now trained with both the ground truth bottle total and our imputed estimate of it (chosen randomly at train-time as an augmentation), and receives with it, for each bottle, a binary input indicating whether that bottle’s total is the recorded or the imputed one. With the imputed total instead of the recorded one, BIOTRAM’s JSD goes from 0.1383 to 0.1403 and its MAE per bottle from 0.729 to 0.804, while the Transformer, the best baseline with the imputed total, goes from 0.1398 to 0.1406 and from 0.755 to 0.778. The imputed total mostly affects the scale of the counts, and BIOTRAM, which uses the total as its count scale, depends more directly on it than the baselines.

The paper now reports results with both totals, describes the imputer, its training and the sensitivity analysis in the new App. T, and states the dependence in the model's description and in the Limitations.

3. Results under the revised protocol

When using the ground truth bottle total, BIOTRAM has the best performance across the five deep models explored on all three of our main evaluation metrics: JSD 0.1383 (baselines 0.1396 to 0.1434), MAE per bottle 0.729 (0.751 to 0.774) and AUC 0.9015 (0.8904 to 0.8965; BIOTRAM is scored on its presence probability, and reaches 0.8979 on its predicted counts). Paired over the 20 runs, its JSD is lower than every baseline with p ≤ 0.05 (by 0.0014 against the strongest, the 1-D CNN, p=0.048), same for its MAE per bottle with p ≤ 0.007. When evaluated with an imputed bottle total, BIOTRAM keeps the lowest JSD (now tied with the Transformer), and the Transformer has the lowest MAE per bottle (0.778 against 0.804 for BIOTRAM). We have updated the paper's text accordingly to reflect these new results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.