acceptodds
Withdrawn from ICLR 2027

Beyond Discrimination: Calibrated Geoprior Fusion for Bioacoustic Monitoring

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Modern bioacoustic foundation models like Perch and BirdNET can identify species with high discriminative accuracy, yet their confidence scores are often uncalibrated and difficult to interpret as probabilities of real-world occurrence. This limits their use for ecological inference beyond threshold-based detection. We leverage a global annotated acoustic dataset (WABAD) to produce calibration priors for an acoustic model, optionally incorporating species-level information. We introduce new methods of fusing the acoustic predictions with geopriors, which empirically improves calibration while preserving discrimination. Together, these results suggest a path to simpler and more reliable acoustic monitoring for broad biodiversity.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Bioacoustics, Model Calibration, Geographic Priors, Passive Acoustic Monitoring, Species Distribution Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YknQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposed a method to improve the calibration of existing pretrained foundation models in bioacoustics, in particular for birds. This method aims to remove the need for labeled calibration data at specific monitoring sites to tune thresholds to accept a model's prediction. It uses pre-calibrated scores learned from a global bird dataset and fuses them with available geopriors.

Critical strengths
  • the problem is very important and not investigated enough in the field. a better calibrated model to eliminate the need for location-specific threshold tuning is essential for future ecological monitoring.
  • the method seems to be inexpensive, elegant and easy to apply to existing models, improving model calibration
Critical weaknesses

Small contribution, limited evaluation and lacking related work: This work only evaluated two supervised pretrained CNN models on two datasets. Since the method contribution itself is rather small (a combination of existing methods/models) and the field is pretty niche, this work requires a broader evaluation to be accepted at a top-tier conference such as ICLR in my opinion. At least a broader discussion of the field in related work would be essential (the bare minimum of different foundation models or datasets or the reason why things couldn't be used) for the general audience to get a bigger picture of this domain and put it into proper context. The related work section is very limited with only a few works mentioned. For instance, a more general evaluation on the well-established Birdset benchmark (on different geo location) with available geo information would help strengthen the contribution and better validate the generalization across different environments. The same applies for a broader evaluation of different training approaches.

While I understand that this paper is more about off the shelf models, they are also widely used with frozen embeddings (and e.g., a linear head) and it cannot be ignored entirely. An self-supervised pretrained model as an alternative representation learning technique such as BirdAVES or BirdMAE would be a good case study to strengthen the evidence for generalization across bioacoustic foundation models. Even if the evaluation use case would require some labeled data, this is pretty common in a real-world use case (e.g. for not-supported species) and the threshold-tuning is still a problem.

Also a deeper evaluation with a more specific biological use case to showcase (or discuss it for the genreal audience) the practical need for threshold tuning to accept a model's prediction would be very helpful to strengthen the contribution of this work and make the problem better understandable for non-experts.

Questions for authors

See above in weaknesses, they are also the key questions that affect my assessment. While this could theoretically also be assigned to future work, I think this is essential for the niche-topic paper to be accepted at a general top-tier conference such as ICLR

Reviewer_ig9oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors have used an existing algorithm, named Mean Likelihood Platt Scaling (MLPS), to calibrate a statistical machine learning model for bird species classification from audio, and to incorporate geographical data as side information

Critical strengths

The authors address a challenging and timely problem, that of bioacoustic classification at the global scale. They work on multiple state-of-the-art models (BirdNET and Perch) and on large datasets (WABAD and more). They experiment with several algorithms for model calibration. The paper is very clear and the use of mathematics (Bayes theorem) is appropriate.

Critical weaknesses

(1) My main concern with the paper is novelty. The authors do not invent a new method or algorithm for multimodal machine learning, information fusion, or model calibration. Instead, they use Platt scaling, which is already well-established. Moreover, although they occasionally report gains, these gains are not consistent across models and datasets. I think that this paper deserves to be published but i do not see ICLR as a suitable venue for it.

(2) My second concern is that the authors evaluate models at the instance level (i.e., bird vocalizations), not at the level of "downstream quantitative tasks" (borrowing the authors' language in their future work section) such as call density estimation.

Questions for authors
  • To what extent does this article advance the state of the art in representation learning, either fundamental or applied? Beyond avian bioacoustics, what is the "lesson learned" for practitioners of machine learning at large?
Reviewer_LfeGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method to generate calibrated probabilities for bioacoustic foundation models without the need to collect any additional expert validation data. Their method is based on utilizing existing priors from WABAD, which’s a large global annotated dataset, followed by fusing the calibrated scores with a geographic prior.

Critical strengths
  1. The method is cost-effective and easy to implement as it doesn’t require expensive expert data or any re-training. Training is simple as the authors only fit an affine map on the logits. This is useful because expert validation is the main bottleneck in large scale acoustic monitoring.

  2. The paper introduces a principled approach of fusing scores and a smart utilization of geospatial data.Unlike prior work that simply multiplies the classifier and geoprior probabilities which the authors noted that it pushes outputs to zero and make them underconfident.

Critical weaknesses
  1. Lack of novelty. The method feels like combining existing ideas in the field and applying them on this specific use case because averaging the Platt scaling parameters is basically one global rescaling of the logits, and the fusion formula is the standard Bayes rule.

  2. Limited evaluation and unsupportive results. First, only two datasets were used and one of them was used for validation. Second, the results don’t seem to fully support the introduced method. For instance, if you compare the KL scores for the pre-calibration scores between Table 1 and 3 at the P columns and the MLPS rows, they show that adding NBSP made the KL divergence worse for both Perch (0.063 to 0.072) and BirdNET (0.043 to 0.054). Third, no confidence intervals are reported, so many of the smaller differences in the tables could be noise.

Questions for authors
  1. (Suggestion) What happens if you use wrong or different priors? How much is your method sensitive to the prior statistics?

  2. I am confused about the footnote about Doohan A2O dataset that says “we expect this dataset to be made publicly available soon”. Is it going to be public as part of this paper, or is it independent?

  3. (Suggestion) Can you show stronger evidence that your method improves the results with confidence intervals?

  4. (Suggestion) Would you be able to add more datasets to support your method claims? If no more datasets exist perhaps some synthetically generated datasets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.