VegBench: Quantifying the Ecological Resolution of Geospatial Models with Expert Field Surveys

We introduce *VegBench*, the first geospatial benchmark to focus on how well models recover hierarchical categorization of ecosystems, from coarse groupings (i.e. forest vs. desert) down to fine-scale vegetation types useful for diverse ecological tasks like protecting threatened species and mitigating wildfire. *VegBench* standardizes decades of expert-collected vegetation classification and accuracy assessment field surveys across the state of California, spanning 42,393 surveys of 387 vegetation alliances. We utilize *VegBench* to compare modern geoFMs against remote sensing and environmental covariate baselines, and find that even the best current models can capture broad ecological structure but fall short as ecological resolution approaches the fine-grained distinctions required for conservation decision-making. *VegBench* offers the geospatial AI community a much needed testbed to assess whether and when models capture *ecologically-meaningful* variation.