acceptodds
Under review as a conference paper at ICLR 2027

EarthMind: Persistence-Anchored Latent Transitions for Robust Multimodal Earth Observation Forecasting

Abstract

Forecasting Earth observations is hampered by a practical fact: the inputs are routinely incomplete. In our globally distributed benchmark, only 90.3% of Sentinel-2 months and 82.6% of Sentinel-1 months are usable. We present EarthMind, a multimodal spatiotemporal world model that frames monthly forecasting as learning a transition from a persistence-anchored state rather than regenerating the scene from scratch. EarthMind fuses Sentinel-2 multispectral imagery, Sentinel-1 SAR, static terrain, and ten weather variables into latent Earth states. It predicts the next state with a Transformer and decodes that state into future Sentinel-2 and Sentinel-1 imagery and weather. Two extensions target real-world use. Structured modality masking (EarthMind-Robust) reproduces realistic sensor outages during training. Band-wise calibration toward persistence (EarthMind+) learns how much of the predicted change each spectral band can be trusted with. On 1,000 sites with site-disjoint splits (16,562 test sequences), EarthMind, trained from scratch, performs comparably to the pretrained Prithvi-EO-2.0 foundation model on clean inputs, with no significant difference on any metric, while additionally forecasting SAR and weather. EarthMind+ attains the lowest RMSE (0.0676), highest PSNR (23.40 dB) and SSIM (0.9704), and lowest SAM (4.68) among the evaluated methods, while Prithvi retains the lowest MAE. When SAR and weather are both missing, structured masking reduces RMSE by 35.3%. In four-month closed-loop rollouts, EarthMind stays 42.8% below last-observation-carried-forward, and its frozen latent state improves land-cover balanced accuracy by 12.2 points over raw features without any land-cover supervision. The learned calibration coefficients further identify the shortwave-infrared bands as the least predictable.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.