Autoregressive Embedding Prediction for Temporal Earth Observation Foundation Models

Multi-spectral remote sensing time series contain rich temporal and spectral signals. However, existing remote sensing foundation models often struggle to learn robust temporal semantics from these data due to their sensitivity to observation-specific noise and insufficient modeling of fine-grained temporal structure. We propose AREO, a feature-predictive foundation model that addresses these limitations by learning semantic representations while preserving fine-grained temporal structure throughout the prediction process. We introduce Autoregressive Embedding Prediction, in which a causal predictor, combined with temporally consistent high-ratio masking, estimates each masked patch's representation from current and preceding context. This design avoids noisy observation-level reconstruction and instead encourages temporally stable, spatially localized semantic representations. Pretrained on SSL4EO-S12, AREO is evaluated under both frozen-backbone and full fine-tuning protocols across three temporal and two static datasets. It outperforms the baselines on the temporal benchmarks under both evaluation protocols and achieves competitive results on static tasks. These findings suggest that temporally directed embedding prediction can support transferable representations for Earth observation.