acceptodds
Withdrawn from ICLR 2027

Leveraging Ecological Context in Biological Foundation Models at Scale

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Understanding the natural world often requires more than visual information: where and when an organism is observed provides critical evidence about its identity. Yet biological foundation models are trained predominantly from images and text, without exploiting readily available ecological context. We introduce a framework that incorporates geographic location and time of day and year directly into the semantic space of biological vision–language models. Using a novel dataset of more than 50 million multimodal observations, we train an ecological context encoder aligned to the BioCLIP 2 embedding space and an adaptive fusion mechanism combining ecological and visual evidence learned per-observation. Unlike other species-prior approaches, our method does not require a fixed species-specific classification head, preserving a flexible text-based retrieval interface in which candidate taxa are represented through their text embeddings. Across held-out species-recognition benchmarks, including novel biodiversity-weighted evaluation sets designed to better reflect biodiversity-rich regions underrepresented by observation effort, ecological context consistently improves recognition performance when combined with visual representations. Improvements are largest in geographically under-observed regions where visual training data are comparatively sparse. Our results demonstrate that large-scale ecological metadata provides a complementary modality for biological foundation models and offers a scalable route toward geographically robust biodiversity recognition.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Contrastive learning, multimodality, biology, ecology, biodiversity, geospatial

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_a9y1OpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores additional modalities to visual and textual modalities in ecology, namely geographic location, time of year, and time of day, which unlike other species-prior approaches, does not require a fixed species-specific classification head, preserving a flexible text-based retrieval interface in which candidate taxa are represented through their text embeddings. The authors also collect a dataset comprising approximately 50 million images over 151.7K species which was aligned with the pretrained BioCLIP-2 taxonomic text embedding space. Consistent with existing literature, the results demonstrate that ecological context provides complementary modality for biological foundation models.

Critical strengths

It is encouraging to see recently more AI research addressing relevant challenges in biodiversity monitoring and ecology. A particular strength is the paper's attention to how ecological context can improve species recognition in biodiversity-rich but underrepresented regions where visual training data are limited.

Critical weaknesses

Overall, I find the contribution of this work rather limited. The central idea of aligning ecological, visual, and textual representations is not novel, as this was already explored in TaxaBind [1] and CrypticBio [2]. The proposed framework incorporates only coordinates, day of year, and time of day, overlooking potentially important environmental variables such as climate, elevation, and habitat characteristics, given that previous work, such as TaxaBind [1], has already explored richer multimodal ecological representations.

Comparing the ecological-to-image alignment versus the ecological-to-text alignment while keeping the backbone, training data, and fusion architecture identical would directly test whether the new strategy, i.e., observation-specific gated fusion, provides a meaningful improvement to TaxaBind's alignment strategy.

Also, the original dataset is not yet properly referenced, i.e., DOI missing (line 753-754). Neither the dataset nor the trained model appears to be publicly available at this stage of the submission. Authors should also reference all datasets in Table 1.

However, my main concern is the evaluation. The authors rely solely on reacall @1 and @5, which provide a rather limited assessment of model performance. For example, mean reciprocal rank might be more relevant for human-in-the-loop systems.

[1] Sastry, Srikumar, et al. "Taxabind: A unified embedding space for ecological applications." 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2025.

[2] Manolache, Georgiana, Gerard Schouten, and Joaquin Vanschoren. "Crypticbio: A large multimodal dataset for visually confusing species." Advances in Neural Information Processing Systems 38 (2026).

Questions for authors

1.Do the authors have any insights on how current SOTA VLMs would perform on the task of species classification?

2.I am curious how would the proposed fusion mechanism perform on visually challenging cases involving morphologically similar or cryptic species, using benchmarks like CrypitcBio?

3.An important question I have which have not been discussed in the paper: does the quality of photograph taken play a role in misidentification of species (for example the species being far away from the camera and being almost invisible, typical for birds even research graded within iNaturalist)?

4.Could the authors extend the evaluation, such as mean reciprocal rank and F1-score?

2 comments

We sincerely thank the reviewer for their thoughtful assessment and constructive feedback. We address each concern below.

W1 Method novelty and additional environmental modalities.

We thank the reviewer for this observation. We acknowledge the important foundations established by previous multimodal approaches, e.g. TaxaBind, and agree that our methodological contributions relative to these works warrant clearer emphasis. Our approach differs through the joint representation of geographical, seasonal, and diel information, the use of taxonomic text embeddings as the initial alignment target for ecological context, and the investigation of multimodal learning at substantially greater observational and taxonomic scale.

Regarding additional environmental modalities, we agree that information such as climate, elevation, and habitat characteristics could provide valuable complementary evidence. However, our primary objective is to investigate the utility of routinely available observational metadata, which is inexpensive to collect and already widely available across large-scale biodiversity datasets. This enables us to study ecological context at foundation-model scale without requiring additional data sources.

We believe that demonstrating the feasibility and practical utility of this approach at scale represents a valuable contribution to scientific applications of AI, independently of the methodological novelty of individual components. We will endeavour to make these motivations and distinctions clearer in future versions of the manuscript.

W2 comparison of ecological-to-image and ecological-to-text alignment

We thank the reviewer for this suggestion and agree that a comparison of alternative alignment strategies would help clarify the contribution of our methodological choices.

Our motivation for aligning ecological context directly with taxonomic text is to learn associations between ecological context and taxonomic identity independently of visual appearance, as we understand that these can be complementary for identification.

We acknowledge that our current comparisons do not isolate the effect of the alignment target from differences in backbone and training data. We agree that investigating ecological-to-image alignment under otherwise identical experimental conditions would be a valuable direction for future research.

W3 Dataset availability and reference

We thank the reviewer for raising this point. The DOI for the underlying dataset was redacted as to not break anonymity during the review process. We apologise for any confusion, and acknowledge that this could have been communicated more clearly.

We will make all code and dataset processing scripts available with future versions of this work.

We also appreciate the suggestion to provide explicit references for datasets in Table 1, and will address this in future versions of the manuscript.

W4 Additional evaluation metrics

We thank the reviewer for this suggestion. We agree that R@1 and R@5 provide a limited characterisation of retrieval performance, particularly for applicatitons where users may be “in the loop”, inspecting multiple candidate species. We acknowledge that additional ranking metrics would provide a more comprehensive assessment of model utility in these cases. We intend to extend our existing evaluation set to include these in future work.

Q1

We thank the reviewer for this question. Our evaluation already includes several established biological vision–language models that we consider representative of the current state of the art, including BioCLIP, BioCLIP 2, BioTrove-CLIP, and BioCAP. We would appreciate further clarification regarding which additional models or comparisons the reviewer has in mind, as this would help us better address their question in future work.

Q2

We thank the reviewer for this suggestion. We agree that morphologically similar and cryptic species represent a particularly relevant application for ecological context, as geographical and temporal information may help distinguish taxa that are difficult to identify from appearance alone.

However, as discussed in response to other reviewers, established datasets such as CrypticBio contain occurrences overlapping with BioCLIP 2 pretraining data, which makes comparisons misleading. This is why we therefore prioritised constructing our own dedicated evaluation partitions with explicit controls for pretraining overlap.

Q3

We thank the reviewer for raising this interesting question. We agree that image quality is likely to influence species identification. This is an important motivation for incorporating ecological context: when visual evidence is ambiguous or insufficient, geographical and temporal information may provide complementary evidence for taxonomic identification. While we have not explicitly evaluated this aspect, we will endeavour to make this motivation clearer in the introduction in future versions of the manuscript.

Q4

We agree that additional metrics, particularly mean reciprocal rank, could provide a more informative assessment of retrieval performance in practical applications. As noted in our response to W4, we aim to extend our evaluation beyond R@1 and R@5 in future work.

We thank the reviewer again for their constructive assessment and for raising several important questions regarding the practical applications and evaluation of our framework.

Reviewer_k5iTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a multimodal framework that incorporates geographic coordinates, day of year, and time of day into the semantic embedding space of BioCLIP 2 for improved taxonomic recognition. A multiscale Fourier-feature encoder learns ecological representations aligned with taxonomic text embeddings using contrastive learning, followed by a second training stage that jointly optimizes ecological, visual, and fused representations through a learned scalar gating mechanism. The authors construct a large-scale dataset of approximately 50 million image–taxonomy–metadata triplets spanning 151.7K taxa and introduce biodiversity-weighted evaluation benchmarks. Experiments demonstrate substantial improvements over image-only BioCLIP 2, particularly for amphibians, birds, and mammals in biodiversity-rich regions.

Critical strengths
  1. The problem formulation and motivation is meaningful and relevant. Integrating ecological context is highly relevant to ecological recognition, especially when visually similar species occupy distinct geographic ranges or exhibit seasonal activity patterns.
  2. The authors explicitly address geographic sampling imbalance, observation-level splitting, and overlaps with BioCLIP 2 pretraining data. They introduce a large dataset with 151k taxa for this.
  3. Good improvements over the baselines are reported in the multimodal setting along with geographically aware evaluations.
Critical weaknesses
  1. The method and the motivation itself are limited in novelty since several works have already explored integrating multimodal data for species recognition. The paper misses BioViTA [1] as a potential related work.
  2. The main weakness of the paper is that the proposed multimodal training degrades image-only performance of the model as compared to the frozen BioCLIP-2 backbone as seen in Table-3. I have a strong suspicion that text-metadata alignment objective has caused the text to drift away from image. As a controlled ablation, the authors could try BioViTA or TaxaBind's pretraining approach.
  3. The motivation for fusing ecological metadata into the visual representation is insufficiently justified. Since the text modality primarily consists of taxonomic labels with limited semantic variability, ecological metadata arguably provides more useful information for conditioning the candidate taxonomic representations than for modifying the visual embeddings. We can clearly see the evidence as image-text alignment has worsened in the experiments.
  4. If the method clearly degrades image-only performance of BioCLIP2, then why not integrate metadata through a spatio-temporal prior as done in previous works [2]. The proposed method in the paper closely resembles [2] but with a jointly trained spatio-temporal prior.
  5. The paper has limited benchmarks and most are limited to the proposed dataset. The paper should atleast report results on established benchmarks like CrpticBio to convince that the framework can generalize.
  6. The paper does not report any results on generalization to unseen taxa or dataset.
  7. Although the authors have created a biodiversity weighted bechmark, there is no benchmark for spatial generalization. The paper should explore spatially disjoint train and test sets to establish the method can generalize well to unseen geographic locations.
  8. Can the framework handle and detect spatio-temporal bias in the training? Certain geographic locations and times are frequently sampled than the others. How is this bias handled in the training? For this the authors could try masking approaches like MIAM [3].

References

[1] Shinoda, Risa, et al. "BioVITA: Biological Dataset, Model, and Benchmark for Visual-Textual-Acoustic Alignment." arXiv preprint arXiv:2603.23883 (2026).

[2] Mac Aodha, Oisin, Elijah Cole, and Pietro Perona. "Presence-only geographical priors for fine-grained image classification." 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2019.

[3] Zbinden, Robin, et al. "MIAM: Modality Imbalance-Aware Masking for Multimodal Ecological Applications." The Fourteenth International Conference on Learning Representations. 2026.

Questions for authors
  1. Why did the authors choose to fuse metadata to the visual embeddings rather than the textual embeddings?
2 comments

We thank the reviewer for their careful consideration and constructive feedback. We appreciate their recognition of the importance of ecological context for biological recognition, the scale of our dataset, and the improvements demonstrated by our geographically aware evaluations. We address each concern below.

W1

We thank the reviewer for this observation. We acknowledge the important foundations established by previous multimodal approaches and agree that our methodological contributions relative to these works warrants clearer emphasis. In particular, our approach differs through the joint representation of geographical, seasonal, and diel information, the use of taxonomic text embeddings as the initial alignment target for ecological context, and the investigation of multimodal learning at substantially greater observational and taxonomic scale.

We also thank the reviewer for highlighting BioVITA, which we agree is an important contribution that should be discussed alongside existing multimodal approaches. We will endeavour to clarify these distinctions and expand the related work in future versions of the manuscript.

W2

We thank the reviewer for highlighting this important concern. We acknowledge that multimodal adaptation reduces image-only retrieval performance on some benchmarks. However, we note that our visual encoder remains frozen throughout training, meaning that these differences arise from changes to the shared semantic space rather than degradation of the visual representations themselves. We agree that alternative alignment strategies, including those explored by TaxaBind and BioVITA, warrant further investigation, and hope to explore these approaches in future work.

W3

We thank the reviewer for this observation and agree that the motivation for our fusion strategy could be more clearly articulated. Here, we view image and context as providing complementary sources of evidence for taxonomic identity. So, we align the context encoder to the text embeddings, rather than vision.

We agree that alternative strategies, including conditioning candidate taxonomic representations on ecological context, represent interesting directions for future investigation.

W4

We thank the reviewer for this discussion point. We agree that geographical-prior approach proposed [1], provides important motivation for our work. However, a central design decision in our framework has been to preserve the text-based retrieval capabilities of CLIP-based models. Conventional neural priors, with species specific classification heads are not easily extendable in this way and require more parameters as the number of species increases. The method proposed here produces fixed-dimensional representations within the existing semantic space, which we argue is easier to scale.

Nevertheless, we agree that comparison with conventional geographical priors is important. In future work, we will endeavor to include such comparisons by training a geographical prior network on our dataset. We believe this will provide a stronger characterisation of the relative strengths of these approaches.

W5 and W6 We thank the reviewer for raising these important concerns. We agree that evaluation on established external benchmarks is valuable to assess generalisation. However, datasets such as CrypticBio and TaxaBench draw substantially from the sources of the training set used for our approach and BioCLIP~2, it has not been possible to fairly evaluate the model on these sets, as contamination from train to test risks inaccurate evaluation.

For this reason, we have prioritised developing a dedicated evaluation partitions, with splitting performed with BioCLIP2’s training set and with the data in our corpus. We apologise that this has not been made clear, and we will ensure to make this clear in future versions of the manuscript.

We also agree that generalisation to unseen taxa is an important direction for future evaluation. We acknowledge this limitation and hope to investigate taxonomically disjoint evaluation in future work.

W7

We thank the reviewer for this suggestion and agree that evaluating generalisation to spatially disjoint regions is an important direction for future work.

Our primary objective in this study is to investigate whether ecological context provides complementary information for taxonomic recognition at scale, rather than to evaluate geographical extrapolation to entirely unseen regions. We acknowledge that our existing evaluation does not establish spatial generalisation and hope to investigate this more thoroughly in future work.

W8

We thank the reviewer for raising this consideration. We agree that spatiotemporal sampling bias represents a challenge when learning from biodiversity observations.

As detailed in Appendix A.3, we address the prominent sampling biases identified during dataset construction through spatial and temporal downsampling. Specifically, we cap observations at 10,000 per 1° × 1° geographical cell to reduce the dominance of heavily sampled regions, and downsample concentrated temporal observation events to mitigate associated sampling artefacts. While these procedures cannot eliminate all underlying observational biases, we believe that the careful construction and processing does represent a contribution in its own right.

We also thank the reviewer for highlighting MIAM, which addresses the related challenge of modality imbalance during multimodal training. We agree that investigating such approaches could provide useful further improvements and represents an interesting direction for future research.

Why fuse metadata with visual rather than textual embeddings?

We thank the reviewer for this question. Our motivation is to combine two complementary sources of evidence for taxonomic identification: vision and context. This reflects the motivation of prior work (e.g., [1]), which learns species priors independently of visual information and subsequently combines these with visual predictions. As we adopt a similar perspective within the shared taxonomic embedding space, we will endeavour to make this motivation clearer in future versions of the manuscript. We thank the reviewer again for their thorough assessment and for raising interesting points, which we believe will help strengthen future versions of our work.

[1] Mac Aodha, Oisin, Elijah Cole, and Pietro Perona. "Presence-only geographical priors for fine-grained image classification." 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2019.

Reviewer_S2FhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a large-scale biodiversity dataset containing over 50 million observations with images, taxonomic labels, geographic locations, and temporal information. The authors also propose a framework that integrates ecological context into the embedding space of BioCLIP 2 and combines visual and ecological features through an adaptive gating mechanism. Experimental results show that incorporating ecological context improves species recognition performance.

Critical strengths
  • Incorporating geographic information into biological foundation models is an important research direction.
  • The study uses large-scale biodiversity observations with geographic and temporal metadata.
  • The paper includes useful ablation studies.
  • The biodiversity-weighted evaluation is interesting.
Critical weaknesses
  1. Inconsistent dataset statistics. In Appendix A.3, the authors report 28,960,850 occurrences after filtering, while Table 4 reports 29,214,714 occurrences in total. In addition, the reported training counts appear to include occurrences that should have been reserved for validation and testing. These inconsistencies need clarification.

  2. Insufficient comparison with simple geographic priors. Table 3 shows that ecological context alone provides some predictive information. However, the large improvements from fusion may simply come from using geographic information to narrow down the species candidates predicted by the image model. Comparing BioCLIP 2, BioVITA (CVPR'26) , and the proposed model with simple species priors (e.g., species occurrence probabilities estimated from the training set) would help clarify whether the proposed semantic alignment is necessary.

  3. Limited benefit of adaptive gating. In Table 3, the simpler ungated fusion outperforms the proposed gated fusion on 9 out of 16 metrics, including both R@1 and R@5 on the standard test set. These results raise questions about the necessity and effectiveness of the proposed adaptive gating mechanism.

Questions for authors

As mentioned in the weaknesses, could the authors clarify the inconsistencies in the dataset statistics? Also, would simply incorporating geographic species priors estimated from the training set into existing image models achieve similar improvements?

1 comment

We sincerely thank the reviewer for their thoughtful and constructive feedback. We particularly appreciate their recognition of the importance of incorporating geographical information into biological foundation models, and their positive assessment of our dataset, ablation studies, and biodiversity-weighted evaluation. The review highlights several important areas where our methodology and experimental comparisons could be strengthened, which we address below.

W1 We thank the reviewer for identifying these inconsistencies and apologise for the error in our reported dataset statistics. We acknowledge that the occurrence counts presented in Appendix A.3 and Table 4 are inconsistent due to an error in our reporting. We appreciate the reviewer bringing this to our attention and will ensure that these discrepancies are corrected and the dataset statistics are clearly documented in future versions of the manuscript.

W2 We thank the reviewer for this suggestion. We acknowledge that estimating geographical species priors from the distribution of the train set is an interesting and direct approach. This has been discussed in previous works [1]. It can become computationally expensive and difficult to scale for large datasets and taxonomies, as we have here. In future versions, we will make this motivation for our work more clear as we think this distinction warrants clearer emphasis.

W3 (see also response to W6, reviewer JH2B)

We thank the reviewer for highlighting the limited advantage of our gating network. We agree that the results we presented did not consistently establish a benefit over fixed-weight fusion.

We aim to investigate revised fusion approaches in future work, as we believe it to be an important aspect of our framework.

We also thank the reviewer for their questions, which we believe are adequately addressed in our responses above.

[1] Mac Aodha, Oisin, Elijah Cole, and Pietro Perona. "Presence-only geographical priors for fine-grained image classification." 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2019.

Reviewer_JH2BOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes to leverage geolocation and time metadata to facilitate biological foundation model learning. Specifically, the authors design an ecological context encoder and train it to align with species taxonomic labels. Fusing the ecological information with visual features leads to significant improvement in species classification, especially for under-represented geolocations.

Critical strengths
  1. Fusing context information into the visual features is a reasonable route, compared with aligning them directly, and I believe it would be a promising way to improve biological foundation models.
  2. The proposed model achieves significant performance improvement in species classification, especially in biodiversity-weighted benchmarks, demonstrating better practical use of biological foundation models.
  3. The visualizations (e.g., Figure 4) are interesting. I also appreciate the transparency in showing some errors and noise in the visualized map.
Critical weaknesses
  1. The authors claim that “unlike other species-prior approaches, our method does not require a fixed species-specific classification head.” However, previous CLIP-based approaches such as BioCLIP and BioTrove don’t use a classification head either but employ a similar retrieval-based inference. Please be more specific here in terms of “other species-prior approaches,” or correct this claim.
  2. The ecological–text alignment stage trains the context encoder to produce embeddings aligned with BioCLIP 2 text encoder. I am not sure if this alignment is reasonable, as a single species might have a very broad distribution, and subspecies can have clear geographical gaps, but using the same taxonomic labels for different geolocations might discard the fine-grained differences and create conflict when the same geolocation is aligned with different species.
  3. This paper proposes an ecological context encoder, and the authors train it to align with taxonomic labels. Another potential solution is to use some existing context encoders, such as GeoCLIP and SatCLIP, as initialization. These encoders have more independent training objectives and are less biased by species/observation distribution, and they reflect the local ecosystem given a geolocation. It is unclear whether the proposed ecological context encoder provides clear advantages over previous ones.
  4. While not identical, the overall structure is similar to Taxabind, but using the text embeddings as anchors instead of image embeddings. The proposed work also uses a much larger training set and BioCLIP 2 as initialization, compared with BioCLIP in Taxabind. I will not suggest that this is an unfair comparison, but the authors are encouraged to better highlight the difference from Taxabind, the methodological novelty, and why these components can provide better guidance in model training.
  5. The evaluation of location–taxonomy retrieval is confusing. Intuitively, it is not feasible to retrieve target species using locations only, as the same geolocation can support a diverse range of species. If the target species appears directly as the top-1 retrieved species, it can only suggest that this location has a strong correlation with that species in the training set, potentially due to observational bias. I would think R@50 or R@100 to be more reasonable metrics, where the target species appears within a certain range of the retrieved list.
  6. The authors propose to use a trainable gate to balance the visual and context branches. However, Table 3 does not show a clear advantage for the gated version. Figure 1 (a) shows that the proposed model has better performance for southern latitudes, but there is no analysis of the necessity of the trainable gate.
  7. While a biodiversity-weighted benchmark can better reflect the practical use of these foundation models, there is the concern that the geolocations with limited observation collections also have more human bias in data collection. For example, one geolocation might be tightly connected to one or two certain species. Therefore, while this effort of creating more “realistic” benchmarks is well acknowledged, it remains unclear whether the proposed benchmark is indeed more realistic or not. For example, it would be helpful to have some justification for the species diversity of less-represented geolocations in the test sets.
Questions for authors

The authors curate biodiversity-weighted benchmarks. Why not a biodiversity-weighted training set as well?

2 comments

We sincerely thank the reviewer for their thoughtful and constructive assessment of our work. We particularly appreciate their recognition of the potential of ecological context for biological foundation models, and of the importance of evaluating these models in biodiversity-rich distributions. The comments have been very helpful in identifying areas where our methodology, evaluation, and discussion could be strengthened. We address each point below.

W1 We thank the reviewer for highlighting this ambiguity and apologise for the imprecise wording in our claim. We intended to distinguish our approach from conventional species-prior methods, such as neural geographical priors, which typically rely on fixed, species-specific classification heads. We did not intend to suggest that this distinguishes our approach from CLIP-based models such as BioCLIP and BioTrove, which already support text-based retrieval. Our contribution is instead to extend this flexibility to ecological context by aligning it with the existing taxonomic space. We will be sure to clarify this better in future versions of the work.

W2 We appreciate the reviewer's concern regarding the potential limitations of aligning geographical context with species-level taxonomic representations. We agree that species can occupy broad geographical ranges, and that taxonomic supervision alone may not adequately capture geographical variation within species or differences between subspecies. However, in this work, our objective is not to learn species distributions explicitly, but to capture the associations between ecological context and species ID, so that context can complement visual evidence as a prior. We recognise that the contrastive objective may introduce competing supervision where multiple taxa share similar ecological contexts, which is an important limitation of this formulation. This is an aspect we aim to investigate in further work.

W3 We thank the reviewer for their suggestion and agree that investigating pretrained geographical encoders would be a valuable direction for future research. The primary focus of our work has been to investigate the contribution of routinely available observation metadata which is available at great scale. We have therefore focussed on learning the ecological representation directly from this information, without integrating additional encodings derived from other sources.

We recognise, however, that pretrained geographical representations may offer complementary information and potentially improve the our ecological encoder. We have not claimed here that our initialisation strategy is superior and hope to explore these alternatives in future work.

W4 We thank the reviewer for bringing to our attention the need to better distinguish our methodology from TaxaBind. We agree that the work is significantly different to TaxaBind due to the training method presented, the addition of season and diel information into a shared encoding, alongside geographic features, and due to the substantially greater scale of our endeavor, aiming to align with the scale of biological foundational models.

We also acknowledge that the difference in dataset scale and pretrained backbones also contribute to performance differences, in future versions of this work we will make this distinction more clear.

W5 We appreciate this point and agree that R@1 and R@5 provide a limited characterisation of context-only performance in this case. We agree that higher-recall metrics such as R@50 and 100 would help to improve this aspect of the manuscript and provide a more intuitive measure. We intend to incorporate these into future evaluations.

W6 We thank the reviewer for highlighting the limited advantage of our gating network. We agree that the results in our submission did not consistently demonstrate a benefit over fixed-weight fusion.

We aim to investigate revised fusion approaches in future works, as we agree that the ability to adaptively select the contribution from vision or context is a critical strength of the work.

W7 We thank the reviewer for raising this important consideration. We agree that biodiversity-weighted sampling does not eliminate the observational biases inherent within the GBIF, and we do not intend to suggest our benchmarks fully resolve this. Our objective has been to construct complementary evaluation distributions, which place greater emphasis on important regions, which have been independently identified as having high biodiversity richness. Nevertheless, we agree that the resulting benchmarks are still constrained by the availability of underlying observations. We will clarify this limitation much better in future versions of the manuscript and hope to extend our analysis of taxonomic and geographical characteristics in future work.

The authors curate biodiversity-weighted benchmarks. Why not a biodiversity-weighted training set as well?

We thank the reviewer for this question. Our primary motivation for introducing biodiversity-weighted evaluation sets is to establish benchmarks that better represent biodiversity-rich regions and can support future research. We therefore deliberately focus on biodiversity-weighted evaluation rather than training, allowing us to assess how models trained on large-scale observational datasets perform under these alternative distributions. We hope these benchmarks will provide a useful basis for comparing future approaches-- possibly including those that explicitly incorporate biodiversity-aware sampling during training. We agree that investigating such training strategies is a promising direction for future work.

Many thanks again to the reviewer, their feedback has been valuable and we appreciate their time and expertise on this matter.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.