acceptodds
Under review as a conference paper at ICLR 2027

GeoState: Pretrained Geospatial Embeddings Are Forecastable Where the Land Changes

Abstract

Geospatial foundation models are evaluated almost exclusively as static feature extractors. We ask whether their representations also behave as predictive states: can next year’s embedding of a location be forecast from its recent history? We study annual 64-dimensional AlphaEarth Foundations embeddings over five Indonesian regions. We forecast the residual with extrapolation, linear, MLP, GRU and Transformer forecasters under matched budgets and five seeds. We evaluate them under temporal and geographic holdouts with spatial-block bootstrap intervals. On the primary 2024 test, aggregate gains over persistence are modest and no architecture clearly wins. Stratifying by change explains why. Learned forecasters lose slightly to persistence at stable locations but gain substantially at dynamic ones, including when change is identified from information available at forecast time. The apparent geographic attenuation disappears once differences in dynamism are accounted for. A non-foundation control shows that aggregate error can mislead. PCA, autoencoder and quarterly representations of Sentinel-2 are easier to forecast overall, but their skill is not consistently concentrated at land-cover change and brings little downstream benefit. AlphaEarth instead shows a strong, model-consistent association between forecast skill and observed land-cover change. Its predictable component is state-conditional and anti-persistent, consistent with mean reversion, and it varies across years. Aggregate forecast error alone is therefore a poor test of whether a representation is a useful predictive state.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Geospatial Foundation Models, AlphaEarth, Geospatial Embeddings, Spatiotemporal Forecasting, Representation Learning, Earth Observation, Remote Sensing, Land-Surface Change, Geographic Generalization, Predictive State Representations

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ZavkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks whether a fixed, independently pretrained geospatial representation can itself be forecast as a future state. Using annual 64-dimensional AlphaEarth embeddings at approximately 16,000 locations in five Indonesian regions, the authors predict the next-year embedding from recent embeddings and compare persistence, velocity extrapolation, ridge regression, MLP, GRU, and Transformer forecasters. Aggregate gains over persistence are modest, but gains concentrate at changing locations.

Critical strengths
  1. Uncertainty is treated seriously. Five training seeds, paired spatial-block bootstrap intervals, block-size sensitivity checks, and separate results for the held-out regions make the reported effects more credible.
  2. Comprehensive analysis based on AlphaEarth.
Critical weaknesses

The practical value of forecasting the embedding is not yet established, and the evidence for generalization to consequential changes is narrow. The paper evaluates the predicted embeddings against the true next-year embeddings and uses them for land-cover classification and NDVI regression. However, these results do not show that forecasting the embedding is more useful than directly predicting next-year land cover or NDVI from information available at forecast time. In the held-out regions, every learned forecaster achieves a lower land-cover F1 than the persistence baseline. Also, the authors should demonstrate a practical task in which forecasting the embedding provides a clear advantage (i.e. compared with directly predicting the remote sensing images or the land cover change). Besides, the study does not distinguish gradual or recurrent land changes from exogenous shocks such as wildfire, typhoon, tsunami, or policy-driven land conversion. These cases may have different predictability from embedding history alone, so the current evidence does not yet support a broad operational interpretation of change-localized forecastability.

Questions for authors
  1. Why forecast the embedding, and how should its value be judged? What concrete use case benefits from forecasting the embedding before next-year imagery is available? AlphaEarth inference from next-year imagery defines the target but is not available at forecast time, a more relevant comparator may be a direct outcome forecast or a forecast-image-then-encode pipeline. On the same train/test splits, how does a probe applied to the forecasted embedding compare with a model trained to predict next-year land cover or NDVI directly from the current and past embeddings? The authors should demonstrate whether the predicted embedding is right in a practically meaningful sense.
  2. What happens under event- or intervention-driven distribution shifts? Could the authors identify how many sampled locations were affected by wildfire, typhoon, tsunami, or major policy-driven land-use change, and report performance separately where counts permit?
  3. Are the elements of Figure 1 data-based or illustrative? Please clarify the provenance of the “Sentinel-2 NDVI” panel, which appears schematic, and verify that the Indonesia map correctly shows the island outlines and sampled study regions. If any panel is AI-generated or conceptual, please label it as an illustration and distinguish it from measured NDVI or georeferenced data.
  4. How does annual sampling affect the model comparison? I am positive about forecasting geospatial foundation-model embeddings, but a one-year step may be too coarse to reveal trajectory information (i.e. yearly is quite longer, but if you predict monthly or even weekly, with several historical inputs, it would be more possible). Likewise, ridge/MLP/GRU/Transformer are useful controlled baselines, but the absence of change-aware, spatial-context, or exogenous-input designs limits the breadth of the conclusion.
Reviewer_j2SVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether embeddings from pretrained geospatial foundation models can serve not only as descriptions of the current state but also as predictive states, using embeddings from previous years to forecast the next year’s embedding. Using annual AlphaEarth embeddings from five Indonesian regions, the authors compare persistence, extrapolation, ridge regression, MLP, GRU, and Transformer forecasters under both temporal and geographic holdout settings. Overall, the paper addresses a valuable question and presents a carefully designed experimental study, particularly through systematic checks of random-seed variability, spatial dependence, duplicate data, differences in dynamism distributions, and simple representation controls. However, some central claims remain stronger than the available evidence supports, especially those concerning actual land change, geographic transfer, and the underlying mechanisms.

Critical strengths
  1. Most existing geospatial foundation models are evaluated as static feature extractors, whereas this paper directly investigates whether pretrained embeddings themselves exhibit predictable dynamics. This question has potential relevance to forecasting future land change and to research on representations as states. The research question therefore offers a degree of novelty and practical significance.

  2. The authors provide a reasonably comprehensive comparison of forecasting architectures. Ridge regression, MLP, GRU, and Transformer achieve broadly similar gains in dynamic areas, and most differences between models lack clear statistical support. This shifts the conclusion from “complex sequence models are better at predicting change” toward “this predictability is primarily a property of the representation rather than an advantage of a particular architecture,” which is more credible and more generalizable. The statistical treatment is also careful. Recognizing the substantial spatial autocorrelation among remote-sensing pixels, the authors use a spatial block bootstrap rather than computing confidence intervals through simple pixel-level resampling. They also examine robustness across different block sizes, with the main conclusions remaining largely unchanged. This reduces the risk of overstating statistical significance due to spatial pseudoreplication.

Critical weaknesses
  1. The main signals used for semantic evaluation and change stratification are Dynamic World and NDVI, both of which are derived from Sentinel-2, while AlphaEarth itself also incorporates Sentinel-2 data. These experiments therefore demonstrate that AlphaEarth’s predicted changes are associated with Sentinel-2-derived remote-sensing change signals, but do not yet establish that they correspond to actual land changes observed independently. The authors explicitly acknowledge this limitation and state that an independent evaluation using Landsat-based Hansen forest loss has been implemented but remains incomplete. I believe this independent validation should be included in the current paper, as it directly bears on whether the title and the central “land change” claim are justified.

  2. The oracle dynamic stratum is defined directly using the realized change ∥Zt+1−Zt∥\lVert Z_{t+1}-Z_t\rVert, while the error of persistence naturally increases with the magnitude of that change. Consequently, the finding that persistence performs worse in the dynamic stratum is partly built into the stratification scheme. The authors address this concern through stratification based on past change and Dynamic World label changes, and these two analyses are actually more convincing than the oracle stratification. I suggest presenting observable-change and label-change stratifications as the primary evidence in the main text, while treating oracle stratification mainly as a diagnostic analysis.

  3. The appendix shows that the Transformer trained with MSE alone achieves substantially higher MSE skill than the default MSE-plus-cosine setting, both in-domain and under OOD evaluation, without a clear reduction in dynamic-stratum gains. Since MSE skill is one of the paper’s primary evaluation metrics, retaining the comparatively weaker default loss in the main experiments requires a more thorough explanation. A more natural approach might be to use MSE-only training as the main setting and treat the cosine term as an ablation, or explicitly state that the training objective was fixed before examining the test results.

Questions for authors
  1. Could the authors complete the independent validation using Hansen/Landsat forest loss in the current version? With change labels independent of Sentinel-2, do AlphaEarth’s forecasting gains remain significantly concentrated at locations undergoing change?

  2. Is the AlphaEarth embedding for year tt generated exclusively from data available by the end of year tt? Could temporal interpolation, smoothing, or gap-filling incorporate observations from t+1t+1 or later years? Please clarify the temporal cutoff for the information used to construct each annual embedding.

  3. How were the five study regions selected? Were these regions and their boundaries determined before examining the extent of land change or the model results? Could the authors provide more explicit region-selection criteria and the complete sampling frame?

  4. Why do the main experiments use MSE plus cosine as the default neural training objective rather than the MSE-only setting, which achieves substantially better MSE skill in the appendix? Was this choice fixed before examining the test results? Would the main conclusions remain unchanged if all primary experiments used MSE-only training?

  5. There are currently only two independent held-out regions, and Papua is nearly static. How do the authors define the scope of the geographic transfer claim supported by these experiments? Have they considered adding more geographically held-out regions or further narrowing the claims about geographic generalization?

  6. The in-domain results for 2023 differ markedly from those for other years. Could the authors further examine whether embedding displacements, land-cover transition types, or external environmental conditions exhibited systematic anomalies in 2023, to help explain why all learned models failed in that year?

Reviewer_cbwPOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether AlphaEarth embeddings contain forecastable temporal structure. The authors compare learned forecasters against persistence in five regions in Indonesia, evaluating latent-space accuracy, downstream semantic retention, and change-stratified forecasting skill. They find that learned forecasters for AlphaEarth outperform persistence when land cover changes, while that pattern does not hold with compressions of raw Sentinel-2 data.

Critical strengths

• Interesting question about the predictive capabilities of pretrained geospatial foundation model embeddings

• Multiple evaluation perspectives, including latent accuracy, downstream semantic retention, and change-stratified analysis

• In-domain and geographically held-out evaluations provide useful evidence about the transferability of the findings

Critical weaknesses

• Lack of engagement with related work and related models and embedding products. AlphaEarth is not the only source of geospatial foundation model embeddings, other embeddings (and even full models) like OlmoEarth, TESSERA, Prithvi-EO, Clay, etc. would be useful to compare in this work. The Sentinel-2 representations offer an interesting baseline, but comparisons with other pretrained geospatial embeddings would help establish whether the observed forecasting advantage is specific to AlphaEarth or generalizes across geospatial foundation model representations.

• Additionally, engagement with and discussion of the nuance of AlphaEarth’s training is limited. While there is not an explicit autoregressive forecasting objective described in the AlphaEarth paper, the embeddings are designed to capture temporal structure. Additionally, AlphaEarth uses a variety of maps and geo-located supervision sources in the training process, including NLCD, which is a land cover product.

• The manuscript needs substantial editing. For example, there is a lack of support and motivation for claims in the introduction and related works, terms from the methods section appear in the introduction before being defined, figure 1 is difficult to read and contains information that is not immediately relevant.

Questions for authors

• The paper’s reproducibility statement claims that code is provided in the supplementary materials, but I could not locate the materials in the submission. Could you confirm where I can find these resources?

• Could the authors provide more motivation for the three evaluation categories? What additional insights does each provide?

• Given the weaknesses noted above, could the authors discuss how AlphaEarth's training objectives and supervision sources may influence the observed forecastability? Why were other pretrained geospatial embeddings not considered?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.