- Rating
- 3
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper explores additional modalities to visual and textual modalities in ecology, namely geographic location, time of year, and time of day, which unlike other species-prior approaches, does not require a fixed species-specific classification head, preserving a flexible text-based retrieval interface in which candidate taxa are represented through their text embeddings. The authors also collect a dataset comprising approximately 50 million images over 151.7K species which was aligned with the pretrained BioCLIP-2 taxonomic text embedding space. Consistent with existing literature, the results demonstrate that ecological context provides complementary modality for biological foundation models.
Critical strengths
It is encouraging to see recently more AI research addressing relevant challenges in biodiversity monitoring and ecology. A particular strength is the paper's attention to how ecological context can improve species recognition in biodiversity-rich but underrepresented regions where visual training data are limited.
Critical weaknesses
Overall, I find the contribution of this work rather limited. The central idea of aligning ecological, visual, and textual representations is not novel, as this was already explored in TaxaBind [1] and CrypticBio [2]. The proposed framework incorporates only coordinates, day of year, and time of day, overlooking potentially important environmental variables such as climate, elevation, and habitat characteristics, given that previous work, such as TaxaBind [1], has already explored richer multimodal ecological representations.
Comparing the ecological-to-image alignment versus the ecological-to-text alignment while keeping the backbone, training data, and fusion architecture identical would directly test whether the new strategy, i.e., observation-specific gated fusion, provides a meaningful improvement to TaxaBind's alignment strategy.
Also, the original dataset is not yet properly referenced, i.e., DOI missing (line 753-754). Neither the dataset nor the trained model appears to be publicly available at this stage of the submission. Authors should also reference all datasets in Table 1.
However, my main concern is the evaluation. The authors rely solely on reacall @1 and @5, which provide a rather limited assessment of model performance. For example, mean reciprocal rank might be more relevant for human-in-the-loop systems.
[1] Sastry, Srikumar, et al. "Taxabind: A unified embedding space for ecological applications." 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2025.
[2] Manolache, Georgiana, Gerard Schouten, and Joaquin Vanschoren. "Crypticbio: A large multimodal dataset for visually confusing species." Advances in Neural Information Processing Systems 38 (2026).
Questions for authors
1.Do the authors have any insights on how current SOTA VLMs would perform on the task of species classification?
2.I am curious how would the proposed fusion mechanism perform on visually challenging cases involving morphologically similar or cryptic species, using benchmarks like CrypitcBio?
3.An important question I have which have not been discussed in the paper: does the quality of photograph taken play a role in misidentification of species (for example the species being far away from the camera and being almost invisible, typical for birds even research graded within iNaturalist)?
4.Could the authors extend the evaluation, such as mean reciprocal rank and F1-score?
2 comments
Q1
We thank the reviewer for this question. Our evaluation already includes several established biological vision–language models that we consider representative of the current state of the art, including BioCLIP, BioCLIP 2, BioTrove-CLIP, and BioCAP. We would appreciate further clarification regarding which additional models or comparisons the reviewer has in mind, as this would help us better address their question in future work.
Q2
We thank the reviewer for this suggestion. We agree that morphologically similar and cryptic species represent a particularly relevant application for ecological context, as geographical and temporal information may help distinguish taxa that are difficult to identify from appearance alone.
However, as discussed in response to other reviewers, established datasets such as CrypticBio contain occurrences overlapping with BioCLIP 2 pretraining data, which makes comparisons misleading. This is why we therefore prioritised constructing our own dedicated evaluation partitions with explicit controls for pretraining overlap.
Q3
We thank the reviewer for raising this interesting question. We agree that image quality is likely to influence species identification. This is an important motivation for incorporating ecological context: when visual evidence is ambiguous or insufficient, geographical and temporal information may provide complementary evidence for taxonomic identification. While we have not explicitly evaluated this aspect, we will endeavour to make this motivation clearer in the introduction in future versions of the manuscript.
Q4
We agree that additional metrics, particularly mean reciprocal rank, could provide a more informative assessment of retrieval performance in practical applications. As noted in our response to W4, we aim to extend our evaluation beyond R@1 and R@5 in future work.
We thank the reviewer again for their constructive assessment and for raising several important questions regarding the practical applications and evaluation of our framework.
We sincerely thank the reviewer for their thoughtful assessment and constructive feedback. We address each concern below.
W1 Method novelty and additional environmental modalities.
We thank the reviewer for this observation. We acknowledge the important foundations established by previous multimodal approaches, e.g. TaxaBind, and agree that our methodological contributions relative to these works warrant clearer emphasis. Our approach differs through the joint representation of geographical, seasonal, and diel information, the use of taxonomic text embeddings as the initial alignment target for ecological context, and the investigation of multimodal learning at substantially greater observational and taxonomic scale.
Regarding additional environmental modalities, we agree that information such as climate, elevation, and habitat characteristics could provide valuable complementary evidence. However, our primary objective is to investigate the utility of routinely available observational metadata, which is inexpensive to collect and already widely available across large-scale biodiversity datasets. This enables us to study ecological context at foundation-model scale without requiring additional data sources.
We believe that demonstrating the feasibility and practical utility of this approach at scale represents a valuable contribution to scientific applications of AI, independently of the methodological novelty of individual components. We will endeavour to make these motivations and distinctions clearer in future versions of the manuscript.
W2 comparison of ecological-to-image and ecological-to-text alignment
We thank the reviewer for this suggestion and agree that a comparison of alternative alignment strategies would help clarify the contribution of our methodological choices.
Our motivation for aligning ecological context directly with taxonomic text is to learn associations between ecological context and taxonomic identity independently of visual appearance, as we understand that these can be complementary for identification.
We acknowledge that our current comparisons do not isolate the effect of the alignment target from differences in backbone and training data. We agree that investigating ecological-to-image alignment under otherwise identical experimental conditions would be a valuable direction for future research.
W3 Dataset availability and reference
We thank the reviewer for raising this point. The DOI for the underlying dataset was redacted as to not break anonymity during the review process. We apologise for any confusion, and acknowledge that this could have been communicated more clearly.
We will make all code and dataset processing scripts available with future versions of this work.
We also appreciate the suggestion to provide explicit references for datasets in Table 1, and will address this in future versions of the manuscript.
W4 Additional evaluation metrics
We thank the reviewer for this suggestion. We agree that R@1 and R@5 provide a limited characterisation of retrieval performance, particularly for applicatitons where users may be “in the loop”, inspecting multiple candidate species. We acknowledge that additional ranking metrics would provide a more comprehensive assessment of model utility in these cases. We intend to extend our existing evaluation set to include these in future work.