acceptodds
Under review as a conference paper at ICLR 2027

Adapting AI Weather Foundation Models to a Changing Climate

Abstract

AI weather foundation models learn from historical archives, while the climate distribution encountered during deployment continues to evolve. This growing model–climate mismatch motivates continual adaptation without repeated full-model retraining, but the structure of weather forecast errors determines how such adaptation should be performed. We develop an online adaptation framework that exploits two complementary error structures: geographically persistent biases and flow-dependent errors that vary with the atmospheric state. Analysis of Aurora-1.3B reveals strongly concentrated forecast-loss sensitivity in its decoder, motivating flow-dependent corrections within fixed low-dimensional sensitivity subspaces. A residual memory tracks persistent errors remaining after this correction, while a variable-wise online gate controls the contribution of the persistent component. We evaluate Aurora-1.3B using ERA5 from 1959–1968 for offline initialization and a strictly chronological 1969–2022 stream for online evaluation, with updates restricted to previously available verification targets. For 2-metre temperature, distributional departure from the historical reference, measured directly from ERA5, is associated with larger frozen-model errors and greater adaptation gains, while the adapted model maintains more stable skill across evaluation periods. Across 69 output fields and five lead times from 6 to 72 hours, our method reduces MSE and MAE by 4.17% and 2.76%, respectively, relative to -Adapter, the strongest of six online adaptation baselines. Evolving climate distributions create the need for continual adaptation, while weather-specific forecast-error structure guides how that adaptation should be performed.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
weather forecast, online learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aVRpOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies online adaptation of a weather forecasting model over a long chronological stream. It combines updates in historically sensitive model directions with a memory of persistent forecast errors and an adaptive gate. The evaluation reports average improvements over online adaptation baselines and examines how frozen-model error and adaptation gains relate to departure from the historical climate distribution.

Critical strengths
  1. The evaluation covers a long period and a broad set of outputs. The appendix includes regressions as well as favorable cases, and the reported aggregate improvements can be recovered from the tables. That is useful reporting, even though internal arithmetic consistency does not independently verify the forecasts.

  2. Persistent error correction and state-dependent adaptation are a sensible combination for this application. The variable-wise analysis helps explain why one correction policy would not suit every field. The gating comparison also gives some evidence that always applying residual memory would be a poor choice for particular variables.

Critical weaknesses
  1. The chronological deployment claim needs a clearer experimental contract. Appendix C does not make the initialization source sufficiently explicit: it matters whether the backbone was trained from scratch or adapted from weights with later data exposure. The multi-step forecast/update timeline is also unclear. I cannot tell from the description whether an issued forecast keeps its parameters fixed, how corrected states are fed back, or when each verification target becomes available. Hyperparameter selection and the gate's stated timescale raise related questions. I am not asserting leakage, but these details are necessary to establish the claimed historical separation.

  2. The additional value of the sensitivity-based mechanism is not isolated. Gradient concentration and moving activation patterns are interesting diagnostics, but they do not show that these directions improve adaptation more than a simpler parameterization. The important frozen-backbone-plus-residual-memory control is also missing from the reported ablation. Consequently, persistent bias correction could explain a substantial part of the gain. The aggregate improvement is encouraging, but its variation across fields and time needs uncertainty estimates and simpler controls before the central mechanism claim becomes convincing.

Questions for authors
  1. What exact weights initialize the offline model, and what data periods were used before the stated chronological evaluation?
  2. During an issued multi-step forecast, are parameters fixed? Please describe how forecast feedback, delayed verification, and overlapping initialization times are handled.
  3. When were adaptation hyperparameters selected, and was that selection separated from the online evaluation stream?
  4. What gate normalization, clipping, and forgetting rules were actually used? How does the routing timescale in Appendix C relate to the cumulative update in Equation (11)?
  5. Are correction-only and matched simple-adapter results available from existing runs? If so, can their gains be reported with uncertainty that respects temporal dependence?
3 comments

Weaknesses

W1: Chronological Deployment Protocol

see Q1,Q2,Q3,Q4

W2: Isolating the Sensitivity Mechanism

We agree that gradient concentration and moving activation patterns alone do not establish the benefit of the selected directions. Beyond the six online adaptation baselines, we therefore compare against LoRA and a parameter-matched random-subspace control. At the six-hour lead time, Ours reduces average MSE by 5.58% relative to LoRA and by 2.11% relative to Random Subspace. The latter uses the same adapted layers, coefficient parameterization, and trainable parameter count, directly supporting the value of selecting forecast-sensitive directions over random directions.

W3:Frozen-backbone residual-memory control.

We compare Ours with a frozen Aurora backbone using a seven-day EWMA residual correction at the six-hour lead time. Ours reduces average MSE by 6.53% relative to the residual-memory control, indicating that residual correction alone does not account for the full improvement.

MethodAverage MSE ↓
Frozen Aurora + residual memory217.237
Ours203.041

Questions

Q1: Initialization and Data Provenance

As described in Appendix C.1, we use ERA5 data from 1959–1968 for offline pretraining, followed by chronological online adaptation over 1969–2022. During the pre-training phase, we trained the model for 100 epochs to ensure convergence, and then loaded the checkpoint from the end of pre-training using online initialisation.

Q2: Multi-Step Forecasting and Update Timeline

The model parameters for the multi-step forecast/update timeline remain unchanged; in fact, the model parameters are frozen throughout the entire process.

Q3: Hyperparameter Selection Protocol

Hyperparameter search was conducted separately from the online test stream, and the selected settings were fixed before evaluation. Appendix C.3 summarizes the adopted settings. Appendix C.4 reports results across the tested subspace ranks (Table C.1), adapter learning rates (Table C.2), gate learning rates (Table C.3), and residual-memory half-lives (Table C.4).

Q4: Gate Normalization, Clipping, and Forgetting

The seven-day half-life controls the EWMA residual memory in Eq. (7). The gate follows the cumulative exponential-weighting update in Eq. (11), using normalized endpoint losses clipped to [0,1], without discounting its weights. The wording “a seven-day timescale for its running states” in Appendix C.3 is ambiguous, and we will revise it to distinguish the residual-memory timescale from the gate update.

Q5: Control Experiments and Temporal Uncertainty

Existing runs include both a frozen-backbone residual-memory control and a parameter-matched subspace adapter. We compare them with Ours on 39,440 common six-hour forecast initializations across 69 fields. We compute percentile confidence intervals using 10,000 paired circular moving-block bootstrap replicates of monthly errors, with a 12-month block length and forecast-count weighting. The intervals remain above zero for block lengths of 3, 6, and 24 months.

ControlMSE reduction of Ours (95% CI)MAE reduction of Ours (95% CI)
Frozen Aurora + residual memory6.53% [5.88%, 7.22%]3.52% [3.19%, 3.87%]
Parameter-matched subspace adapter2.11% [1.83%, 2.38%]1.00% [0.87%, 1.14%]

Dear Reviewer, thank you for your time and thoughtful feedback. We would appreciate any further comments on our responses and clarifications when you have a chance. Your feedback would be very helpful to us.

Thanks to the author for the rebuttal. After reading the rebuttal carefully, I will raise the score.

Reviewer_h8otOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper discusses learning a correction to an AI weather model online as new data is observed to adapt to a changing climate. I think they basically have an online LORA correction. It's difficult to describe the approach in more detail due to several major inconsistencies in the text. They present strong empirical results using the AURORA forecast model.

Critical strengths

Learning corrections to weather models based on new data from a shifted distribution is an important problem.

The authors compare to several other methods from the literature to assess the efficacy of their approach.

Critical weaknesses

This paper is poorly written and hard to follow. It feels like it was primarily written by AI (consistent with the AI statement).

Online correction of a dynamical weather model has a rich history in the data assimilation literature. In fact, that literature was largely developed around weather modeling. The lack of discussion of this field and how the proposed methods relate is concerning.

There are phrases throughout that are opaque, like "concentrated forecast-loss sensitivity in its decoder", which is never really explained since there's not an explicit decoder here. Another example is "Sequential bias-correction methods use previously verified residuals to reduce systematic regional errors." What does it mean to verify a residual? If it was verified previously, why do we need a sequential method? Is the issue systematic region errors or distribution drift due to climate change? These examples are part of what makes the paper feel AI-generated and underdeveloped.

More generally, the authors say the primary motivation is to adapt predictions to a changing climate. Then they jump to correcting persistent regional biases, which is a totally different thing and doesn't require the online update proposed.

The core idea is "updating coefficients within fixed low-dimensional forecast-sensitivity subspaces", which sounds like an online version of LORA; it would be nice to see this connection described or explored. They also have "a variable-wise online gate [that] controls how much of this persistent correction is added to the forecast." This makes it sound like they learn a correction and then they learn a gate that functions as a correction to the correction. Why is this needed? Why is the original correction insufficient? It's unclear and difficult to follow.

Questions for authors

no questions

3 comments

Weaknesses

W1: Insufficient connection to relevant data assimilation.

We appreciate the connection to data assimilation, but our experimental task differs from conventional state estimation. ERA5 provides the input states and verification targets; we do not assimilate additional observations to update the atmospheric initial state. Instead, we update lightweight adapter coefficients and forecast-bias estimates after verification becomes available.

W2: Unexplained design of correction and gating modules.

The two modules address complementary error structures. The sensitivity adapter learns state-dependent corrections, while the residual memory estimates persistent errors remaining after adaptation. Our additional results support these choices: Ours reduces average MSE by 6.53% relative to frozen Aurora with residual memory.

MethodAverage MSE ↓
Frozen Aurora + residual memory217.237
Ours203.041

W3: Misalignment between motivation and proposed approach.

We distinguish why adaptation is needed from how it is performed. Changing deployment distributions motivate continual adaptation, while persistent and flow-dependent error structures guide the correction design. We do not assume that persistent biases arise exclusively from climate change.

Figure 5 shows that larger T2M distributional departures are associated with higher frozen-model error and greater adaptation gains, supporting mitigation of drift-associated degradation rather than a complete solution to distribution shift.

W4: Figure–Text consistency

We respectfully ask the reviewer to re-examine Figure 1.The fixed bases are labeled UℓU_\ell and VℓV_\ell, where ℓ\ell denotes the network layer, not time tt.Online adaptation updates Aℓ,tA_{\ell,t}, giving ΔWℓ,t=UℓAℓ,tVℓ⊤.\Delta W_{\ell,t}=U_\ell A_{\ell,t}V_\ell^\top. The gate is also explicitly shown as λt\lambda_t in the figure, controlling the residual correction in pt+λtbtp_t+\lambda_t b_t. We respectfully clarify that St(p)S_t(p) and Sℓ,rq\mathcal{S}_{\ell,r}^{q} denote different objects. Equation (3) defines the fixed forecast-sensitivity subspace. St(p)S_t(p) is a scalar projected-state energy fraction, explicitly defined in the colorbar of Figure 3 and explained in its caption.

Reviewer_h8otOpenReview ↗

Thank you

Thank you for your response to my review. I sincerely apologize for misreading Figure 1; it was rendering blurry on my computer, but I zoomed in after reading your response, and you are correct. I have edited my review.

Thank you for revisiting Figure 1 and updating your review.

We appreciate your reconsideration and hope our clarifications have addressed your concerns. We would be grateful if you could also reconsider your overall rating in light of our responses.

Reviewer_gkAPOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors introduce an online adaptation method for ML weather models to correct for drifts. The method makes use of a decomposition of the error into geographically persistent biases and state-dependent errors that depend on the atmospheric state. Sensitive subspaces are identified via Nyström from historical loss gradients and updates are computed from these gradients cheaply.

Critical strengths
  • the first instance of online adaptation to this setting as far as I am aware of
  • the approach is interesting and novel to the best of my knowledge
Critical weaknesses
  • I think LoRA should be cited as the low-rank adaptation is very similar in spirit, despite the differences in subspace selection.
  • the evaluation is limited. How can you be sure that the improvement in error comes from adapting to the distribution shift and not from extra training? I think it would make sense to add as a baseline the same method but with continued training, maybe with LoRA to better understand where the improvement comes from
  • I think the claim that this method takes care of distribution shift is overstated for the above reason. At least it iis not conclusively proved in the manuscript
  • in table 2 it would be good to have the original forecast as a baseline
Questions for authors
  • is this method universal or specific to Aurora. How much of a gain could be achieved with another ML weather model?
  • for your decomposition, can you quantify whether there is a misspecification error when decomposing the error like this?
5 comments

Weaknesses

W1: Comparison with LoRA.

We compare our method with LoRA at rank 128. Ours reduces average MSE and MAE by 5.58% and 2.91%, respectively, relative to LoRA. Both methods adapt the same linear layers while keeping the backbone weights frozen.

MethodAverage MSE ↓Average MAE ↓
LoRA (rank=128)215.0325.1614
Ours (rank=128)203.0415.0111

W2: Continued training.

To assess whether extra training improves generalization, we evaluate the checkpoint on 1969 (out of training years) and after 10 additional training epochs. The weighted L1 loss changes from 0.042680025 to 0.042689971. This indicates that additional training yields no benefits.

W3: Evidence of Distribution-Shift Mitigation.

Our claim is that the method mitigates the performance degradation associated with distribution shift, rather than fully resolves its effects. Figure~5 supports this claim: T2M distributional departure correlates with frozen-model error (r=0.84r=0.84) and adaptation gains (r=0.79r=0.79).Frozen-model MSE rises from approximately 0.300.30 to 0.37 K20.37\,\mathrm{K}^2, whereas Ours remains near 0.25 K20.25\,\mathrm{K}^2.

W4: Comparison with the original forecast.

The tables below reports six-hour MAE and MSE for eight representative fields. Lower is better. Six-hour MAE for eight representative atmospheric fields (lower is better).

FieldOriginal forecastδ-AdapterProceedOneNetFSNetADAPT-ZELFOurs
Z50029.29820.38123.65621.20821.56122.56529.29818.803
T8500.254640.241350.245290.241780.245440.243530.254640.23936
U8500.431650.430150.431180.431140.434020.431550.431660.42938
V8500.436330.435270.436260.436240.439050.436620.436350.43454
T2M0.337990.331300.333560.334150.334220.335410.338000.30100
U100.268710.267740.268560.268560.270200.268690.268730.26647
V100.273570.272480.273490.273510.275210.273600.273580.27147
MSL26.13524.72625.68725.62024.97125.99226.13623.308

Six-hour MSE for eight representative atmospheric fields (lower is better).

FieldOriginal forecastδ-AdapterProceedOneNetFSNetADAPT-ZELFOurs
Z5001409.729676.367915.514733.233762.317842.6271409.867582.499
T8500.129100.120010.122900.120830.123430.121960.129120.11811
U8500.356630.354590.356080.356050.360200.356560.356670.35384
V8500.367020.365390.366880.366860.371180.367320.367060.36485
T2M0.326910.310190.320120.321070.319810.323820.326940.25162
U100.140600.139600.140510.140500.141950.140600.140630.13853
V100.143720.142580.143670.143670.145280.143740.143740.14190
MSL1202.5031088.8521166.7261162.1411111.8571194.6731202.782976.540

Questions

Q1: Applicability Beyond Aurora.

The framework is not tied to Aurora's architecture: it requires access to forecast-loss gradients, suitable weight matrices, and sequential verification targets. The sensitivity bases are estimated separately for each backbone. However, our empirical validation is limited to Aurora, so we cannot quantify gains for another weather model without additional experiments.

Q2: Decomposition Rationale and Misspecification.

Our decomposition is motivated by established approaches in traditional numerical weather prediction. In particular, Danforth et al. (2007) estimate and correct mean bias, periodic errors, and state-dependent model errors. This provides precedent for combining persistent and state-dependent corrections. Our two-component formulation is an operational approximation inspired by this literature, rather than an exact physical decomposition. The residual term accommodates unexplained errors, but our current analysis does not separately quantify structural misspecification from estimation and optimization errors.

Reference Danforth, C. M., Kalnay, E., and Miyoshi, T. (2007). Estimating and Correcting Global Weather Model Error. Monthly Weather Review, 135(2), 281–299.

Dear Reviewer, thank you for your time and thoughtful feedback. We would appreciate any further comments on our responses and clarifications when you have a chance. Your feedback would be very helpful to us.

Thank you for the detailed response and the additional experiments.

W1 / W4. I appreciate the LoRA comparison and the frozen baseline. Please add the LoRA citation in the revised manuscript. Could you also confirm that LoRA was trained online on the same 1969–2022 dataset, with the same update schedule and the same restriction to previously available verification targets? Otherwise the comparison isn’t apples to apples.

W2. I don’t think the new experiment addresses my concern. Training for 10 more epochs on 1959–1968 and evaluating on 1969 shows that more epochs on the original data do not help. My question was different: do the gains come from adapting to a shift, or simply from learning from new data? The method sees 1969–2022, and the training-only control does not. Two controls would separate these:

  • standard online fine-tuning (full or LoRA) on the same chronological data, under the same constraints
  • the proposed method in a stationary setting, e.g. years shuffled or evaluation within the training distribution.

If the gains are comparable, they can't be attributed to distribution shift.

W3. I appreciate the softened claim. However, the evidence in Figure 5 is correlational, and both the T2M departure and the amount of accumulated adaptation grow with time. That makes time a confounder: any method that keeps accumulating corrections would show the same trend. The controls above would also resolve this.

Additional questions raised by the new tables. At 6h, Z500 MSE falls from 1410 to 582 (−59%), and δ-Adapter alone roughly halves it. A gain this large and immediate suggests a substantial systematic bias in the frozen model rather than gradual climate drift. I realize time is limited now, but for future revisions I would include a simple online bias-correction baseline, such as a running-mean bias correction per grid point and variable. This would show how much of the improvement the flow-dependent component adds beyond correcting the mean bias.

To my understanding, Aurora’s pretraining data includes ERA5 from 1979 onward, which overlaps most of the 1969–2022 evaluation. Could you clarify which data the Aurora backbone was trained on and what “offline initialization on 1959–1968” refers to? If the backbone has seen much of the evaluation period, the interpretation in terms of a changing climate becomes difficult.

Q1. Since validation is limited to Aurora, I would suggest narrowing the title and claims accordingly.

Overall, the new results improve the paper, but my main soundness concern remains: the evaluation doesn’t yet isolate adaptation to distribution shift from additional fitting or bias correction. I would also like to apologize for flagging this for Ethics Review - this was accidental and I have corrected this.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of the Discussion and Main Revisions

We thank the reviewers for their feedback. Two reviewers followed up during the discussion. Reviewer gkAP raised their score from 1 to 2 after confirming that the historical data-overlap concern was resolved. Reviewer h8ot acknowledged misreading Figure 1 and amended their review; their follow-up did not report a score change. Reviewer aVRp had not posted a follow-up at the time of this summary.

Reviewer gkAP — Score raised from 1 to 2

  • Historical data overlap. Appendix C.1 already specified the offline training period (1959–1968) and the online evaluation period (1969–2022). We clarified that the Aurora backbone was trained from scratch on 1959–1968 ERA5 data. The reviewer explicitly confirmed that this resolved their concern about exposure to evaluation data and raised their score from 1 to 2.

  • Original forecast baseline. The reviewer asked how much each method improves on the forecast before adaptation. We added the original frozen forecast to the six-hour MSE and MAE tables for all eight selected fields. For Z500, for example, the MSE was 1409.729 for the original forecast, 676.367 for δ-Adapter, and 582.499 for Ours. The reviewer acknowledged this addition and used it to raise the more specific question of how much of the gain simple bias correction could recover.

  • Online LoRA comparison. The reviewer asked whether LoRA had the same opportunity to learn from new data as our method. We confirmed that both methods used the same chronological 1969–2022 stream, update schedule, and restriction to verification targets already available at each update. Both adapted the same linear layers at rank 128. Across 39,440 six-hour forecasts and 69 fields, Ours reduced average MSE by 5.58% and MAE by 2.91% relative to online LoRA. This directly addresses the fairness of the comparison.

  • Bias correction and climate-drift attribution. For Z500, online bias correction reduced six-hour MSE from 1409.729 to 623.095, recovering approximately 95% of the total reduction achieved by Ours. The full method further reduced Z500 MSE to 582.499. Across all 69 fields, its average MSE was 6.53% lower than that of the bias-only control. Figure 5 shows that, for T2M, larger departures from the 1959–1968 climate were associated with higher frozen-model MSE (r=0.84r=0.84) and larger adaptation gains (r=0.79r=0.79). The meteorological literature documents long-term surface warming (IPCC, 2021) and changes in 500-hPa geopotential height (Christidis and Stott, 2015; Simmons, 2022). These findings support the climatic motivation for adaptation, while the fraction of the forecast improvement attributable specifically to climate drift remains open.

Reviewer h8ot — Three misreadings: layer index, residual gate, and subspace notation

  • Fixed bases and online coefficients. The reviewer initially read the bases in Figure 1 as time-indexed. The original figure labels them UℓU_\ell and VℓV_\ell, where ℓ\ell indexes the network layer. Both bases remain fixed during deployment. The time index belongs to Aℓ,tA_{\ell,t}, the coefficient matrix updated online.

  • Gate in Figure 1. The reviewer also reported that the gate was missing. Figure 1 explicitly shows λt\lambda_t in the corrected forecast pt+λtbtp_t+\lambda_t b_t, where it scales the residual correction. After enlarging Figure 1, the reviewer acknowledged having misread it and amended their review.

  • Definition of St(p)S_t(p). The review described St(p)S_t(p) as an undefined subspace. The original submission defines both quantities: Eq. (3) defines the fixed sensitivity subspace Sℓ,rq\mathcal{S}_{\ell,r}^{q}, and Figure 3 and its caption define St(p)S_t(p) as the fraction of state energy projected onto that subspace at location pp and time tt. This scalar varies with the atmospheric state even though the basis remains fixed.

  • Residual correction. Once the ERA5 target is available, we use the forecast residual to update the local bias estimate. A variable-wise gate scales this correction for future forecasts. At six hours, bias correction alone recovers approximately 95% of the total Z500 MSE reduction from the frozen model to the full method. The full method further lowers MSE from 623.095 to 582.499.

  • Connection to data assimilation. We clarified what verification updates in our setting. Once the ERA5 target becomes available, the forecast error informs the adapter coefficients and residual-bias estimates used for subsequent forecasts. The initial atmospheric state of an already issued forecast remains unchanged.

The reviewer’s follow-up explicitly resolved the Figure 1 concern. It did not separately address the definition in Figure 3 or the discussion of data assimilation.

Reviewer aVRp — Score raised from 2 to 3

  • Historical separation and update timing. Appendix C.1 specifies the offline training period (1959–1968) and the chronological evaluation period (1969–2022). We clarified that the backbone was trained from scratch on the historical data and that each online update used only verification targets available at that time.

  • Hyperparameter selection. Hyperparameters were selected separately from the online test stream and fixed before evaluation. Appendix C.3 lists the adopted settings, and Tables C.1–C.4 report the searches over subspace rank, adapter learning rate, gate learning rate, and residual-memory half-life.

  • Residual-memory and gate updates. The seven-day half-life applies to the residual memory in Eq. (7). The gate in Eq. (11) uses cumulative exponential weighting of normalized endpoint losses clipped to [0,1][0,1], with no discounting of past gate losses.

  • Value of the sensitivity directions. The reviewer asked whether the choice of directions improves forecast skill beyond low-rank adaptation alone. We compared the historical sensitivity basis with a random basis while holding the adapted layers, coefficient parameterization, and trainable parameter count fixed. Across 39,440 six-hour forecasts and 69 fields, the sensitivity basis reduced average MSE by 2.11%.

  • Bias-only control and uncertainty. Relative to a frozen backbone with seven-day residual correction, the full method reduced six-hour average MSE by 6.53% (95% CI 5.88–7.22%). This comparison measures the combined contribution of flow-dependent adaptation and variable-wise gating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.