acceptodds
Under review as a conference paper at ICLR 2027

PhenoMap: Phenological Neural Fields for Ecological Mapping

14 views

Abstract

Foundation models in scientific domains have predominantly organized representations around a single dominant axis of supervision. In biology, this axis has been species identity, typically captured via image–text contrastive pretraining or multimodal systems that use species as the binding modality. Yet for plants, visual appearance changes dynamically across developmental stages, a process known as phenology, making species an incomplete signature. Dedicated phenology models reduce this variability to narrow classification targets, while standard taxonomic foundation models encode species identity with bias to phenophases overrepresented in training distributions. To close this gap, we introduce PhenoMap, a multimodal model that leverages time, location, climate, and species to produce factorized species and phenology embeddings aligned with plant imagery. To train and benchmark our model, we curate Pheno3M, pairing citizen-science plant imagery and phenophase annotations with per-observation climate and satellite drivers, and an external expert-survey benchmark. We extensively evaluate PhenoMap on new phenological benchmarks built from Pheno3M, where it excels on compositional species phenophase and fine-grained phenophase retrieval. We further demonstrate its utility by qualitatively showing phenological processes emerging when the spatiotemporal field is queried as a map, and evaluating those learned embeddings on USA-NPN expert observations. All code and data will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Representation Learning, Ecology, AI for sustainability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NwHUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose integrating phenological information into taxonomic foundation models by introducing both a new dataset, Pheno3M, and a new multimodal model, PhenoMap. Pheno3M is a new benchmark dataset that pairs citizen-science plant imagery with phenophase annotations and environmental drivers. By leveraging these annotations with a phenological contrastive loss, PhenoMap produces factorized species and phenology embeddings aligned with plant imagery, outperforming other taxonomic foundation models on phenophase retrieval tasks.

Critical strengths
  • Considering phenology in taxonomic foundation models is a crucial step to make these models really useful across applications, especially with the known bias toward specific phenophases. The factorized approach of PhenoMap is relevant.
  • Proposing both a curated dataset and a well-designed method that outperforms the other taxonomic foundation models. The external evaluation is also commendable.
Critical weaknesses
  • I think the authors could clarify how they expect the produced embeddings to be used and for which ecological applications, and consider evaluation setups accordingly. For example, I am not sure I understand where the mapping suggested by the method name and title fits in. The approach works well to retrieve images of a specific species and phenotype, but its utility to other applications is not sufficiently evaluated. Could it effectively map species distributions at different phenological stages, for instance? I would expect additional experiments in this direction to show its potential for different tasks. Figure 5 is a bit disappointing in this regard: the patterns described are elementary, and I am not sure how they can be useful. The lack of clarity about the true applications of this approach also reflects a field where some models have been called foundation models of biology and/or ecology, even though they were mostly restricted to taxonomic imagery. I believe this paper goes in the right direction, but it needs to clarify its scope.
  • Otherwise, I noticed a few less important points that still deserve clarification and could improve the paper:
    • L22 and L90: “satellite drivers”. These are not the satellites that drive a species' presence or phenophase, but the climate and other factors that satellite imagery may capture.
    • Phenological Contrastive Loss: I think its description could be slightly improved by providing an intuitive explanation.
    • Figure 4 is not discussed in the text. Also, which 2d projection is used?
    • Perhaps I misunderstood something, but in Table 1, I do not expect the retrieval rate to be uniform. In particular, some phenological states make the plant much easier to recognize than others (e.g., at the flowering stage), which would make images from that phenological state more likely to be retrieved. I believe this is rational behavior for the model.
    • Does the image→image setup correspond to querying with another image having the same species and phenophase?
    • The objective of the approach is to perform well across the phenophases. However, there are no results showing that PhenoMap improves performance across phenophases, particularly for those that other baselines are supposed to neglect.
Questions for authors
  • I would like the authors to clarify the intended scope of applications for PhenoMap embeddings and provide additional experiments accordingly.
  • I would like to see my additional points considered and addressed, if necessary.

I am strongly willing to increase my rating if the authors address my concerns appropriately, as I see high value in this paper.

4 comments

We thank the reviewer for the thoughtful review and for recognizing the value of our factorized approach and external evaluation. We address each point below. All section, figure, and table references refer to the revised PDF.


W1, Q1. Scope and the mapping task.

PhenoMap is the first framework to capture the dynamic visual distribution of thousands of species across space and time. By learning continuous, fine-grained representations that capture intra-phase variance, PhenoMap supports a large set of mapping and predictive tasks related to phenology. We evaluate the per-species phenophase mapping task in our experiments with USA-NPN and Pheno3M, where we show that PhenoMap outperforms all baselines in the per-species phenophase mapping. We also demonstrate PhenoMap's extensive qualitative mapping capabilities across species, phenological features, and locations.

Quantitative evaluation of mapping. For each visit, we query PhenoMap’s field branch with the species name, location, and date to generate an embedding without imagery. A probe predicts phenophase activity from these embeddings, which we evaluate against recorded phenological observations. On this task, PhenoMap outperforms BioCLIP 2, TaxaBind and ProM3E (Table 11). This result is especially significant under spatial holdout (0.495 vs. at most 0.359 macro AP). We also show that PhenoMap is superior on held out data from Pheno3M in newly added Table 12. Table 12 evaluates the same phenophase prediction task over 178K observations (per holdout protocol) across 3.3K species, showing that PhenoMap is the strongest phenological predictor on more than USA-NPN data.

To our best knowledge, dense expert-drawn species-specific phenological range maps do not exist, so we use spatially sparse survey data for quantitative evaluation.

Qualitative mapping capabilities of PhenoMap. Additionally, PhenoMap can produce species-specific phenology maps by querying the same dynamic representation of species+time across a spatial grid (Sec. 5.4, App. C). We've added new Figures 16-18 in App. C that show additional qualitative mapping capabilities on flowering, leaf growth, leaf appearance, and more. Figure 7 demonstrates that PhenoMap predicts accurate per-species distributions of phenophases across multiple regions.

W2a. "Satellite drivers." Agreed. We replaced "climate and satellite drivers" with "climate histories and satellite-derived environmental features" in the paper.

W2b. Intuition for the loss. Sec. 4.2.1 now explains that the loss aligns paired image and field embeddings and penalizes similarity between opposite phenophases more strongly than between adjacent ones. To allow within-phenophase variation, only matched pairs are positives masking other samples sharing a phenophase.

W2c. Fig. 4. We have added a discussion of Fig. 4 in Sec. 5.2. We have added the projection method, t-SNE, to its caption.

W2d. Non-uniform retrieval in Table 1. We agree that flowering plants are easier to recognize. But ecological monitoring also needs to detect species when flowers are absent. Otherwise, plants may appear absent simply because they are not flowering. Table 1 gives each species equal numbers of images per phenophase, yet models preferentially retrieve flowering images. To test whether this preference generates worse recognition outside flowering, we report species-identification accuracy separately for each phenophase (App. H.1, Table 19).

The flowering-to-vegetative gap in species-ID accuracy is:

  • 18.3 points for TaxaBind and BioCLIP;
  • 7.5 points for BioCLIP 2;
  • 9.9 points for BioCLIP 2-FT;
  • 6.3 points for PhenoMap, which is also the most accurate model in every phenophase.

W2e. Image→image setup. Yes. Sec. 5.2 (Task setup) defines relevance as matching species and phenophase, for both field and image queries.

W2f. Per-phenophase results. Several results are already broken down by phenophase: retrieval rates (Table 1), and retrieval confusions (Table 27). App. H adds two more breakdowns.

  • Retrieval by query phenophase (Table 26, App. H.4). PhenoMap achieves the highest R@10 in every phenophase for both field→image and image→image retrieval. The largest field→image gains over the strongest baseline are on vegetative (+23.4 points) and fruiting (+20.8 points) queries.
  • Species identification by phenophase. See W2d.

I strongly thank the authors for considering all my comments and, in turn, adding numerous new results. While I appreciate this effort, I find it difficult to understand the differences between the new PDF and the previous one, since they haven't been highlighted (e.g., by showing the new elements in blue). Would it be possible to highlight the differences? In particular, it seems that most changes have been made to the Appendix, and I see here a missed opportunity to bring up in the main text one of the very interesting experiments you ran (e.g., Figures 16-18), which could easily replace Figure 5 that I find much less interesting and insightful (but I leave this choice to the authors' discretion).

Thank you for these helpful suggestions. We have uploaded a revised PDF that highlights all changes in blue: revised text in existing sections appears in blue, and entirely new sections, tables, and figures are marked by a blue heading or caption.

We have also moved Figure 16 into the main paper as Figure 8 in Sec. 5.4. We keep Figure 5 because it visualizes the field embeddings themselves, while Figure 8 shows how these embeddings can be used for discriminative phenology mapping.

Thank you for considering these suggestions. I acknowledge the substantial effort the authors have made to improve the paper during the rebuttal phase. I will therefore increase my rating from 2 to 3, although I believe a score of 3.5 would better reflect my assessment, should the AC wish to consider this (I find it unfortunate that the rating scale does not allow for finer granularity).

Reviewer_G7nEOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses an important problem in plant species identification. As plant appearances vary substantially across phenological stages, existing datasets and taxonomic foundation models often exhibit biases toward specific stages. To address this issue, the paper introduces Pheno3M, a large-scale dataset containing 3 million citizen-science plant observations with phenophase annotations. The paper further proposes PhenoMap, a phenology-aware framework that incorporates temporal, geographic, and climatic context by introducing an additional field branch and phenological contrastive loss to learn plant representations across different stages of their lifecycle. Extensive experiments demonstrate state-of-the-art performance across multiple evaluation tasks.

Critical strengths
  1. The paper is well-motivated and addresses a practical problem that existing taxonomic foundation models and datasets have strong biases.

  2. The paper addresses appearance variations across plant growth stages and proposes a large-scale dataset Pheno3M, and a PhenoMap that improve species identification and ecological mapping performance.

  3. The paper presents extensive experiments and demonstrates strong performance across different evaluation settings.

Critical weaknesses
  1. Several important architectural and training details are not clearly explained:
  • The distinction between the static and dynamic encoders in the image branch is unclear. Do they differ architecturally, or do their representations primarily result from different supervision signals?
  • The proposed phenological contrastive loss introduces cyclic weights WikW_{ik}, but it is unclear how this formulation explicitly captures phenological ordering and cyclicity, rather than simply reducing the repulsion between samples from similar stages. The choice of Wik=0.8W_{ik}=0.8 for adjacent stages also lacks justification.
  • Could the author provide more details about the batch-balancing strategy? In particular, the number of species and samples per phenophase within each batch should be clarified.
  1. The contribution of the phenological contrastive loss is not sufficiently established. The formulation appears to reweight negative pairs according to phenophase similarity, but it is unclear how it captures phenological ordering and cyclicity. Moreover, the strong performance of Dynamic SupCon in Table 13 warrants further discussion to clarify the benefits of the proposed loss.

  2. Missing reference of BioCLIP2.

Questions for authors
  1. How does the proposed phenological contrastive loss capture the ordinal and cyclic structure of phenology? The formulation appears to perform phenophase-aware negative reweighting rather than explicitly enforcing cyclic relationships.

  2. The exploited channel fusion combines static and dynamic embeddings (I'm not sure) as s=αsd+(1−α)sss=\alpha s_d+(1-\alpha)s_s, but sds_d and sss_s are not clearly defined. I'm curious about how the weight α\alpha in channel fusion affects the contribution of each branch across evaluation tasks. An analysis of α\alpha would help clarify the complementary roles of the two branches.

  3. Tables 1 and 3 report variants labeled zsz_s and zdz_d, respectively, without clearly explaining their meanings or how these configurations differ from the standard PhenoMap setting.

1 comment

We thank the reviewer for finding the work well motivated and the experiments extensive. All section, figure, and table references refer to the revised PDF. We address questions below.


W1a. Static vs dynamic encoders. The two encoders use the same MLP design on the frozen BioCLIP 2 image embedding, with different output dimensions (Table 15). Their distinct roles come from their supervision. The static representations are aligned with all data from a species, whereas the dynamic representations are aligned with data that represents a species at a dynamic time + place, which distills fine-grained dynamics into the field embedding.

W1b, Q1, W2. Phenological structure and loss comparisons. We clarify that the phenophase-aware negative reweighting is a core mechanism for learning the cyclic structure of phenology. By encouraging representation similarity to correlate with phenologically-grounded annotations, our model transitively learns cyclic relationships between all phenophases. A core distinction of our loss is that it does not treat same-phase observations as negatives, encouraging representations to naturally diversify when the label does not reflect a different phenological state. In contrast, Dynamic InfoNCE and DynamicSupcon treats same phase observations as negatives of each other, which prevents the learning of phenologically grounded representation similarity.

Thank you for helping us clarify a key mechanism in our framework, we believe motivating the cyclic loss more clearly will sharpen the contribution of our work.

W1c. Batch composition. Each round gives every species a budget of 128 samples, split evenly over its observed phenophases: at most 128 / (#phases) per phase, i.e. 32 for a four-phase species. Each phase is further capped at the species' median per-phase count, so a dominant phase cannot spend the budget when the others are sparse. A phase with fewer observations than its cap contributes all of them.

Example: Phyla lanceolata has 7 / 17 / 291 / 17 training images (vegetative / budding / flowering / fruiting). Its median per-phase count is 17, below the 32 budget cap, so each phase is capped at 17 and the round draws 7 / 17 / 17 / 17. Flowering falls from 88% of its raw data to 29% of its sampled data.

Per round, species blocks are shuffled and cut into batches of 8,192. A batch holds about 154 species (mean 53 samples each, max 128). One round is 41 batches. 15k steps are 366 rounds. Resulting phenophase mix:

VegetativeBuddingFloweringFruiting
Training set14.9%11.2%57.4%16.6%
Per batch after balancing17.8%24.8%33.4%24.1%

W3. BioCLIP 2 reference. We have corrected the citation, as noted in the general response.

Q2. Channel fusion. We clarify that we combine the static and dynamic similarity scores with weights of 0.9 and 0.1, respectively. Table 23 compares each channel alone with this fusion. The dynamic channel gives the strongest phenophase NDCG, while fusion gives the best compositional retrieval, where both species and phenophase must match. The comparison shows how the species information in the static channel complements the phenological information in the dynamic channel.

Q3. z_s and z_d in Tables 1 and 3. The subscripts identify the representation being evaluated: z_s is the static species representation and z_d is the dynamic phenology representation (Sec. 4.1, Fig. 3). We have updated the captions to make this more clear. Table 1 uses the static channel to measure the phenophase distribution of species-name retrievals. Table 3 uses the dynamic channel to evaluate phenological retrieval. In the main retrieval tables, PhenoMap rows without a channel subscript use the fused similarity score described in Sec. 5.2.

Reviewer_wrbBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes PhenoMap multimodal model with embeddings not only based on species ID, but also on the phenophase. They show the inherent bias in currently available foundation models, showing that they have a retrieval bias towards developmental phases like flowering and fruiting. It also proposes a new dataset, Pheno3M. It also quietly presents what appears to be a novel variation of a cyclic-ordinal loss.

Critical strengths

Overall I like the idea and I believe the paper identifies a genuine literature gap:

  • The paper adresses a genuine gap in the literature. Indeed, as the authors identify, phenophase identification for plants is usually done in a rigid way, isolated prediction task, with binary supervision, or is often plant specific. So, creating embeddings that correspond to species x phenophase really has the potential to bring advances to studying plant phenology with vision models
  • The paper shows how their approach removes the bias of available models towards developmental phases like flowering and fruiting by introducing supervision across the orthogonal axes of phenophase
Critical weaknesses

While I think the idea is overall great, the paper as submitted is simply missing many details and seems not to be in a state ready for publication. While any one of these omissions could be overlooked in isolation, they come together to present an incomplete picture of the performed work, making it difficult to gauge the full contribution or methodology of the paper. Based on the current unfinished state of the presentation of this work, I currently recommend rejecting the paper.

  • Missing or incomplete references. The paper builds on BioCLIP-2, a foundation model trained on plant species. They also use a TOL-200M dataset for validating some of their claims. The reference for BioCLIP-2 is missing, as is the one for the TOL-200M dataset (while this dataset did come out together with BioCLIP-2, there is an updated version of the dataset which should be clearly indicated in case it was used). To refer to iNaturalist database, which was the source of images for the constructed Pheno3M benchmark, the authors cite (Van Horn et al., 2018), however this paper refers to iNaturalist 2017. The paper appears to actually rely on iNaturalist 2021 but this is not indicated clearly and instead can only be glimpsed from a small entry in the table associated with Fig. 2. To refer to the USA National Phenology Network data, the authors use (Schwartz et al., 2012) which describes the historical development of USA-NPN, however USA-NPN is actively collecting data to this date and no appropriate reference is used for this. You do not explain what a CONUS grid is nor provide a reference. A reference for InfoNCE loss is missing. Moreover, the paper quietly introduces a new "dynamic alignment loss", yet some related losses are only mentioned in the appendix (while some of the newer related losses are not mentioned, e.g. https://proceedings.mlr.press/v306/lee26bd.html).
  • Many details have been left out that would allow for reproduction or independent verification of the constructed benchmarks. For example, you list your USA-NPN visit-level benchmark as having about 3M observations across 1000 plant species. However, the only details you provide are that canonical filters from (Crimmins et at 2017) and (Rosemartin et al 2018) were used, and that you queried the database for (2013-2025, Plantae, North America). Firstly, (Crimmins et at 2017) and (Rosemartin et al 2018) do not appear to establish a canonical set of filters. Downloading this database for the indicated period and region results in ~30M plant records beloning to ~1400 species. Assuming all the plant records by the same individual on the same date mentioning different phenophases constitute a single visit (which I had to assume as it was not explicit), this produces ~4M observations. If counting only observations in which there is at least one positive phenophase, only this gives around ~3M data points. It is also not specified how you selected exactly 1000 species from the 1399 available. A number similar to the one you specify (1049 species) can be obtained if only species with at least 1000 observations are included. However, none of these details are included in the paper and it is therefore unclear which methodology was used to put together this visit-level benchmark.
  • Your experiments are not clearly described. What exactly is provided as a "field" query? Species name + date and coordinates? Or is also the phenophase provided? In brief, the experimental setup description is too compressed and makes it difficult to understand the details of what was evaluated.
  • You state USA-NPN contains expert observations/records. However, the official USA-NPN website https://www.usanpn.org/data/quality states "Observational plant and animal data in the National Phenology Database are collected by naturalists, professional researchers, high school students and park visitors, and more.". I have not found a way to easily filter the observations made by experts from the citizen science observations. Please clarify your claim that USA-NPN constitutes expert observations.
Questions for authors

Primarily I would expect the authors to address all the questions raised in the list of major weaknesses above; particularly the ones related to proper referencing, attribution, and reproducibility of the work.

In addition to those, there are further smaller inconsistencies in the paper:

  • details of the described architecture are missing from Fig. 3 which shows the architecure. For example, the decode in the image branch that reconstructs the embeddings is not shown.
  • Fig 5 caption says the figure is showing the field embeddings over 12 months, but only 6 pictures are provided
  • You claim in 5.3 that "baselines decay sharply as supervision shrinks, while PhenoMap field probe degrades gradually". Based on Fig 6, I don't find the grade of degradation that different.
  • The code and data was not made available during the review process. Therefore it is difficult to verify the procedures used in the paper (and, as mentioned above, many details needed for independent reproduction seem to be missing)
1 comment

We thank the reviewer for finding our central idea compelling and for their thoughtful questions and close engagement with the paper. We address each point below and have revised the paper accordingly.


W1. References and clarifications. We thank the reviewer for highlighting these referencing issues. We have corrected the citations and clarified the relevant descriptions. Specifically:

  • BioCLIP 2 and TOL-200M. We fixed the broken BioCLIP 2 citation key and specify that we used the original TreeOfLife-200M release on which BioCLIP 2 was trained (App. A.10).
  • Data sources. We distinguish the iNaturalist API data used for Pheno3M (App. A.1) from the iNat-21 comparison benchmark (Fig. 2), and cite each source. Sec. 3 now cites the USA-NPN protocol, data product, and documentation.
  • Contrastive losses. InfoNCE is now cited at its introduction in Sec. 4.2. Sec. 2 discusses our objective alongside SupCon, Rank-N-Contrast, and ConOrd, and Table 16 now includes ConOrd in the loss ablations.
  • CONUS. Sec. 4.1 now defines CONUS as the contiguous United States.

W2. USA-NPN benchmark construction. We thank the reviewer for identifying where our benchmark construction needed clarification. Our filtering follows the data-cleaning guidance in rnpn (Switzer et al., 2026) and the USA-NPN product documentation (Rosemartin et al., 2018). App. A.9 now specifies the filtering, visit construction, and labels, with a summary in Sec. 3. We believe these details, together with the full release of all datasets and models, will aid reproduction and strengthen the contribution of our work.

W3. What is a field query? A field query describes the representation of a species at a specific time and place. It consists of a species name, coordinates, and a date. The species text provides the static embedding, while the date and coordinates select the 365-day PRISM climate window and AlphaEarth embedding used by the dynamic branch. Phenophase labels provide supervision during training (Sec. 4.2.1) and are not supplied as query inputs. The field branch is described in Sec. 4.1 and illustrated in Fig. 3, with the two retrieval tasks detailed in Sec. 5.2.

W4. USA-NPN observers. We thank the reviewer for pointing out this distinction. For the main tables in the paper, we did not filter USA-NPN records by observer expertise, and the benchmark includes both volunteer and professional contributions. Our description of these records as “expert observations” was therefore imprecise. We have replaced it with “standardized survey records,” reflecting the shared observation protocol (Denny et al., 2014). The benchmark evaluates transfer from opportunistic photographs to repeated phenological surveys.

USA-NPN’s monitoring-program metadata allows us to identify visits collected by NEON’s professional field technicians, accounting for 32.5% of benchmark visits. We have added an evaluation on this subset (App. H.5, Table 28), where PhenoMap outperforms both baselines across all three splits.

Q1–Q2. Figure clarifications. We have added the reconstruction decoder to Fig. 3 and have corrected the Fig. 5 caption to describe the eight monthly snapshots shown (January–August).

Q3. Decay in Fig. 6. We now quantify this comparison in Sec. 5.3 using the corrected results described in our general response. Reducing the training set from full size to 100K visits gives relative decreases in macro AP of 14%, 13%, and 7% for PhenoMap on the random, temporal, and spatial splits, respectively. The baseline decreases range from 19% to 42% across models and splits (Table 11).

Q4. Code and data. All data, annotations, splits, construction scripts, weights and code will be released upon acceptance. App. A.9 specifies the USA-NPN construction for independent rebuilding.

References

  • Feldman et al. (2018). How training citizen scientists affects the accuracy and precision of phenological data. International Journal of Biometeorology.
  • Fuccillo et al. (2015). Assessing accuracy in citizen science-based plant phenology monitoring. International Journal of Biometeorology.
  • Switzer et al. (2026). rnpn: Interface to the USA National Phenology Network API. R package version 1.4.2.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revisions highlighted

Following Reviewer NwHU's suggestion, the revised PDF now marks all changes made during the discussion period in blue to aid in reviewing. Revised text in existing sections is shown in blue, and entirely new sections, tables, and figures are marked by a blue heading or caption. Most additions are in the appendix, including the USA-NPN benchmark construction (App. A.9), phenology maps and field-only evaluation (Sec. 5.4 and App. C; Figs. 8, 17, and 18; Table 12), and additional experiments (App. H).

Dear Reviewers and AC,

We thank you for the careful reading and constructive feedback during the discussion period. Your comments and discussion have substantially improved the paper, especially the USA-NPN benchmark descriptions, explanation of the phenological contrastive loss, and the evaluation and visualization of PhenoMap's mapping capabilities.

We hope our responses and revisions have addressed the concerns raised, and we welcome any further questions or comments if anything remains unresolved.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.