Modelling Slow and Fast Variables for Subseasonal-to-Seasonal Forecasting

Deep learning models for weather prediction have reached impressive accuracy. Yet, a significant performance and reliability gap remains at subseasonal-to-seasonal (S2S) forecasting horizons, i.e., 15 days to 2 months. To reach high predictive skill in this regime, accurate estimates are required not only for fast atmospheric dynamics but also for slowly changing boundary conditions on the Earth's surface (for instance, sea surface temperature). This makes S2S dynamics a coupled multi-scale system in which not all variables evolve at the same pace. We address this challenge by extending the Earth System Foundation Model (ESFM), a transformer-based model of weather and climate, such that it is able to explicitly handle the different update frequencies. We introduce a Temporal Attention Aggregator (TAA) taking as input samples from arbitrary past timesteps, and devise a mixed-lead-time (mixed-LT) approach encouraging the model to focus on selected variables' timesteps for each time scale. Our extended TAA model for S2S forecasting (TAAM-S2S), trained with adapted block rollout finetuning, improves the skill of the standard ESFM baseline for slow variables, while matching its accuracy for fast variables at S2S time scales. Moreover, TAAM-S2S outperforms DLESyM, AIFS v1.1 and Aurora v1.5 in terms of ensemble metrics for slow variables. It also reaches competitive performance in a rare-event case study, predicting droughts in Sichuan in 2022 and in Amazonia in 2023.