From Annual to In-Season: Uncertainty-Aware Distillation of Precomputed Geospatial Foundation Model Embeddings

Geospatial foundation models (GFMs) have demonstrated strong performance across a wide range of downstream tasks. Globally precomputed GFM embeddings are increasingly being released as ready-to-use geospatial layers. However, these embeddings are typically annual and retrospective, limiting their use for within-year tasks. Generating new embeddings can also be computationally expensive and impractical at large scales. This precludes their use for temporally-sensitive tasks like agricultural monitoring, where embeddings must be available and re-computed frequently within the growing season. We present an uncertainty-aware framework that distills annual embeddings into temporally evolving representations using partial satellite time series observations. Our model predicts a distribution over the annual embedding conditioned on observations available at a given date. As the year progresses, predicted embeddings converge to the annual representation while uncertainty decreases. We show that the resulting embeddings support in-season agricultural monitoring tasks including cropland classification and field boundary segmentation. Student embeddings achieve high cosine similarity to the annual teacher embedding even with one month of data, and downstream performance matches or exceeds the annual teacher in most tasks. Our method offers a lightweight model to extend annual GFM embedding products to in-season agricultural applications.