acceptodds
Under review as a conference paper at ICLR 2027

Low-Loss Temporal Connectivity in Weather Foundation Models

Abstract

AI weather models are usually trained for one fixed forecast step: a 6-hour model outputs only 6-hour forecasts. To obtain a forecast at an in-between horizon such as 4 hours, one currently has to retrain a new model, chain several short-step models, or interpolate their outputs on the grid. Each option is costly, blurs fine spatial detail, or both. This is a real obstacle for downstream systems (ocean, sea-ice, hydrology and air-quality models, and sub-6-hour data assimilation) that need forcing at cadences unrelated to 6 hours. We show empirically that intermediate horizons can instead be recovered directly in a model’s weights. Starting from a public 6-hour global model checkpoint (Aurora), we fine-tune 3-hour and 1-hour versions with an identical recipe. Comparing internal representations with Centered Kernel Alignment (CKA), the 3-hour and 6-hour models are nearly identical, consistent with a shared low-loss region of weight space, while the 1-hour model has moved to a different region. Along the straight line between the 3-hour and 6-hour weights, a single number λselects a forecaster whose effective lead time tracks λ. We verify this at the two interior horizons that hourly ERA5 offers on this segment: at 0.25◦, plain linear weight interpolation (LWI) at 4 and 5 hours beats every grid-space output-interpolation baseline on near-surface variables, with up to about 2×less error energy at mesoscale wavelengths. Against autoregressive chains the trade-off is the expected one: the chain has lower geopotential point error but damps the spectrum, while LWI preserves the observed energy spectrum at one-third of the chain’s inference cost. Spherical-interpolation and Fisher-weighted variants give no improvement. Connectivity is local: it holds on the 3–6 h segment, decays as the horizon gap grows, and fails under extrapolation. The effect reproduces on two further, architecturally different models (Pangu-Weather and GraphCast) and is stable across training seeds. We present this as an empirical characterization of when weight-space interpolation across lead times works and where it fails; its use as forcing for coupled systems or in assimilation is the motivation, not a demonstrated result.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
weather foundation models, linear mode connectivity, weight interpolation, model merging, loss landscape, fine-tuning, AI weather forecasting, centered kernel alignment, representation similarity, parameter space interpolation, foundation model adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7fuUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the weight-space structure of weather foundation models fine-tuned for different forecast horizons, and proposes using low-loss temporal connectivity to construct intermediate-horizon forecasters. The main experiments start from the public 6-hour Aurora checkpoint and further fine-tune 3-hour and 1-hour models. CKA and related analyses are used to examine the internal representational similarity of different lead-time adaptations. For the 3-hour and 6-hour models that remain in a nearby region of weight space, the paper directly performs linear weight interpolation to obtain 4-hour and 5-hour forecasters without separately training models for those intermediate horizons. The phenomenon is further examined on Pangu-Weather and GraphCast, with comparisons against autoregressive chaining and conventional grid-space interpolation in terms of RMSE, spectral fidelity, and computational cost.

Critical strengths

1. The paper addresses a practical problem caused by the fixed forecast steps of weather foundation models. Many current weather models only provide forecasts at a small set of predefined lead times, whereas downstream ocean, sea-ice, hydrology, air-quality, and higher-frequency data-assimilation systems may require atmospheric forcing at cadences that do not match those fixed horizons. Recovering intermediate states without separately training a model for every target horizon therefore has clear practical relevance.

2. In the main Aurora experiments, weight interpolation provides a favorable trade-off between forecast accuracy and spectral fidelity without additional training for the intermediate horizons. At 4 h and 5 h, LWI outperforms the conventional grid-space interpolation baselines on T2m, T850, U10, and V10. The advantage is even more pronounced in meso-α\alpha error energy, where the error is reduced by up to about a factor of two relative to cubic-spline interpolation for some variables.

Critical weaknesses

1. The Related Work omits a directly relevant line of learned temporal interpolation / temporal super-resolution methods. The paper mainly positions itself through weather foundation models, weight interpolation/mode connectivity, and continuous-time forecasting, but does not systematically discuss learned approaches that reconstruct intermediate states between two known time points, such as weather temporal interpolation, spatio-temporal temporal super-resolution, neural-field methods, or other learned frame-interpolation models. This omission is particularly relevant because the paper attributes the weakness of grid-space interpolation to its inability to reconstruct smoothed fine-scale and high-frequency structure, whereas learned temporal interpolation methods are explicitly designed to model nonlinear evolution and recover intermediate details.

2. The cross-architecture validation on Pangu-Weather relies on a fine-tuned endpoint that is substantially weaker than the official model, which limits the practical value of that part of the evaluation. To obtain an interpolable shared basin, the authors fine-tune a new 3-hour anchor from the official Pangu 6-hour checkpoint. However, this model reaches a Z500 RMSE of 22.2 at 3 h, compared with 13.1 for the independently trained official 3-hour model. More importantly, Pangu officially provides 1-hour, 3-hour, and 6-hour models, yet the paper does not directly compare the interpolated models against 4-hour and 5-hour autoregressive chains constructed from these higher-quality official checkpoints. The paper itself also notes that the light-budget Pangu and GraphCast anchors and their interpolants are not competitive in absolute skill with well-tuned native checkpoints. Thus, this experiment provides useful evidence about the conditions under which shared-basin connectivity can emerge, but does not yet establish that such connectivity can be retained while maintaining competitive endpoint forecast skill.

Questions for authors

1. Learned output-space temporal interpolation. The current grid-space interpolation baselines mainly include linear/quadratic interpolation, cubic splines, linear tendency, and persistence. Have the authors considered learned temporal interpolation or temporal super-resolution methods, such as neural-operator-based weather interpolation, motion-aware interpolation, or other spatio-temporal interpolation networks? If not, what was the main reason? Since spectral fidelity is presented as an important advantage of LWI over grid-space interpolation, whether learned output-space methods can similarly recover mesoscale/high-frequency energy would directly affect the scope of this conclusion.

2. Official Pangu-Weather checkpoint combinations. Pangu-Weather officially provides independently trained 1-hour, 3-hour, and 6-hour checkpoints. Have the authors compared the 4-hour and 5-hour interpolated models against autoregressive chains formed from the official 1-hour and 3-hour models, such as 1h→3h and 1h→1h→3h? If not, why was this comparison omitted? This is particularly important because the fine-tuned 3-hour anchor used for interpolation is substantially weaker than the official 3-hour model.

3 comments

The authors thank the reviewer 7fuU for reading the manuscript and for naming its weaknesses precisely. Both are addressed in the revised PDF, and this comment covers the first one.

W1 / Q1: learned temporal interpolation

You are right that this line of work was missing from our Related Work, and our one-line description of FuXi-2.0 made the gap worse: we wrote that hourly output "required a new model" without saying that the new model is a learned temporal interpolator. The revised PDF corrects that sentence and adds a paragraph on learned temporal interpolation (Sec. 2), which covers FuXi-2.0's interpolator, ModAFNO, HourGlass, TemDeep, STint and AtmoRep.

We did not compare with them in the paper for three reasons.

First, they answer a different question. Each of them takes a completed 6 h forecast and reconstructs what lies between two forecast states. LWI is a forecaster: it produces the 4 h state from (t−4 h, t) in one forward pass, without a 6 h forecast. For the uses that motivate the paper (forcing a coupled model, a sub-6 h assimilation cycle) the quantity needed is the state at t+4 h given t.

Second, they cost what LWI removes. The interpolator is a forecaster-sized network that has to be trained: FuXi-2.0 trains it on its own 6-hourly model's outputs; ModAFNO used about 1300 H100-hours on ERA5 pairs and was not applied to a forecast model's outputs; HourGlass is trained on IFS forecast trajectories. This is the cost class of our natively fine-tuned 4h and 5h models, which Table 1 already reports as the upper reference and which lead LWI by 7–33 %. The output-space baselines of Sections 5–6 are the training-free ones, and the revised text says so.

Third, the spectral question. A learned deterministic interpolator does fix what the cubic spline cannot: motion-aware methods displace features coherently instead of blending them. However, it cannot deterministically regenerate structure whose lifetime is shorter than the bracket: there the intermediate state is not a function of the two bracketing states, and a network trained with an MSE or Charbonnier loss returns their conditional mean, which is a smoothed field. HourGlass motivates its probabilistic design by exactly this observation, and ModAFNO reports no spectra. A CRPS-trained interpolator recovers small-scale energy as samples, which is a different object from a deterministic forecast. LWI's spectrum comes from being a forecaster trained against full-resolution ERA5, with no training at the interior horizon and no sampling.

Thus, the comparison with output-space baselines is now scoped to training-free ones: "every training-free grid-space interpolation baseline" in the abstract, and the same scope in Sec. 2 and the conclusion. Against a learned deterministic interpolator trained on our own anchors' 3 h and 6 h forecasts at the anchors' budget, we expect LWI to keep the spectrum and the interpolator to keep a point-error edge. This is an expectation and we have no result for it. It is the one baseline we would add given the compute of a training run, which is beyond the discussion period; Sec. 8 names it as the missing comparison.

Once again, every prior route to an intermediate horizon (Pangu-Weather's additional models, lead-time conditioning, the learned interpolators) trains a network for it. The question this paper asks is whether the horizon is already encoded between two checkpoints that exist.

Changes in the revised PDF: Sec. 2 (corrected FuXi-2.0 sentence, new paragraph "Learned temporal interpolation"); abstract, Sec. 2 and Sec. 9 (scope of the spectral claim); Sec. 8 (the missing baseline).

W2 / Q2: the Pangu-Weather anchor and the official checkpoint chains

You were right on both counts: the fine-tuned Pangu-Weather 3h anchor of the submission (Z500 22.2 at 3 h) was much weaker than the official 3h model (13.1), and the official chains were not evaluated. Thus, we fine-tuned a new 3h anchor from the official 6h weights to reach the official model's skill: Z500 13.8 at 3 h against 13.14. With it the valley is deeper and better placed than before: at 4 h the minimum lies 53 % below the better endpoint and at 5 h 38.5 % below, ε(λ) < 0 at every interior λ, and λ* = 0.35 and 0.70 against the nominal 1/3 and 2/3 (Table 21, Fig. 2b; 64 initializations of 2022). Then we benchmarked the interpolants, at the nominal and uncalibrated λ, against the official 1h→3h and 1h→1h→3h chains you asked for, on 256 initializations (RMSE, Table 22):

Z500T850U10V10T2m
4 h, LWI27.140.4460.4920.4980.620
4 h, chain 1h→3h26.360.4520.5190.5220.622
5 h, LWI28.010.4800.5530.5650.637
5 h, chain 1h→1h→3h27.460.4890.5650.5740.645

LWI is 3 % and 2 % behind on Z500, ahead on T850 and the 10 m winds at both horizons, and level or ahead on T2m, in one forward pass instead of two or three. Sec. 4.1 and Sec. 8 now describe this anchor.

So connectivity and competitive endpoint skill coexist, on Pangu-Weather as on Aurora. On Aurora we fine-tuned a 6h model from Microsoft's released 12-hour checkpoint. At its own horizon it matches the independently trained official 6h checkpoint (Z500 19.1 ± 3.2 vs 18.8 ± 3.9), and interpolating it with the 12h checkpoint gives an interior minimum at all four interior horizons 7–10 h, 22–39 % below the better endpoint on 64 of 64 initializations, with λ* within one grid step of the nominal (T* − 6)/6 (Sec. 4, Fig. 1, Appendix I). Interpolating the official 6h and 12h checkpoints instead crosses a barrier 1.7–2.6× worse than persistence. Once again, what a pair needs to be interpolable is a shared parent, and the anchor does not have to be weak. The GraphCast anchors, which you also named, remain light-budget proofs of concept (Sec. 8); we did not retrain them.

On Aurora the chains you ask for are in Table 1 (1h→3h at 4 h, 1h→1h→3h at 5 h): the chain wins Z500 by 4.8 and 5.4 m²/s² and loses T2m, T850 and the winds (V10 at 5 h is a tie), at two to three times LWI's inference cost.

Changes in the revised PDF: Sec. 4.1, Tables 21–22 and Fig. 2b (the new Pangu-Weather anchor and the official chains); Sec. 8 (anchor quality); Sec. 4 and Appendix I (the 6–12 h segment).

Reviewer_7fuUOpenReview ↗

Post Rebuttal Comment

Thank you for the detailed rebuttal and additional experiments.

W1/Q1: The expanded related work and narrowed claims address part of my concern. However, without a direct comparison, it remains unclear how LWI performs against learned temporal interpolation in terms of spectral fidelity. The conditional-mean smoothing argument does not, by itself, establish a general advantage over deterministic learned interpolators.

W2/Q2: This concern has been substantially addressed. The improved Pangu-Weather anchor and comparisons with official autoregressive chains provide stronger evidence that shared-basin connectivity can coexist with competitive forecast skill.

Overall, I appreciate the substantial improvements, particularly the additional experimental validation. Nevertheless, the remaining limitation regarding learned temporal interpolation leads me to maintain my original score.

Reviewer_ukiaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether weather models adapted from a common initialization to different forecast horizons are connected by a useful path in weight space. Starting from a public 6-hour Aurora checkpoint, the authors fine-tune 3-hour and 1-hour variants and observe that the 3-hour and 6-hour models have highly similar internal representations, whereas the 1-hour model moves to a more distant region. Linear interpolation between the 3-hour and 6-hour weights produces models for the intermediate 4-hour and 5-hour horizons. The phenomenon is reproduced on Pangu-Weather and GraphCast when the endpoint models share an initialization, while interpolation between independently trained official Pangu checkpoints fails. On Aurora, linear weight interpolation outperforms the tested grid-space interpolation methods on several near-surface variables and preserves more spectral detail. However, autoregressive chains achieve lower Z500 error, natively fine-tuned intermediate-horizon models remain consistently stronger, and interpolated models perform poorly under extrapolation, larger endpoint gaps, and multi-step rollout.

Critical strengths
  1. The paper presents an interesting empirical finding and validates it across three different weather-model architectures.
  2. The positive and negative controls are useful, and the paper is transparent about failures under extrapolation, wider lead-time gaps, and repeated rollout.
Critical weaknesses
  1. The main claim is supported by only two intermediate horizons, 4 h and 5 h, while interpolation fails outside the 3–6 h interval and performs poorly over the wider 1–3 h gap.

  2. The practical value appears limited because native 4 h and 5 h models are consistently better, autoregressive chains are better on Z500, and no downstream coupled-model or data-assimilation experiment is provided.

Questions for authors
  1. Why is the continuous lead-time hypothesis evaluated only on the 3–6 h segment, which contains just two interior targets? Can the authors test a segment such as 6–12 h with more intermediate horizons?

  2. Can a criterion based on CKA, weight distance, or endpoint gap predict interpolation success before evaluating the intermediate targets? Has this criterion been tested across multiple successful and failed checkpoint pairs?

  3. Are the pair-specific λ\lambda values calibrated only on validation data, and do they transfer unchanged to the held-out test year? Do different weather variables prefer different λ\lambda values?

  4. Given that native 4 h and 5 h models are consistently more accurate, what concrete deployment setting makes LWI preferable after accounting for the cost of fine-tuning the 3 h endpoint and calibrating λ\lambda?

  5. Since official Pangu checkpoints include 1 h and 3 h models, why are the 1h+3h and 1h+1h+3h chains not evaluated at 4 h and 5 h?

3 comments

The authors thank the reviewer ukia. We believe we did not articulate well enough the finding we want to report. From our perspective the main message is that it is not at all obvious that mixing the weights of two models of the very same architecture could give a new meaningful model. In the case of language models such a phenomenon somewhat makes sense: the known precedent interpolates along a stylistic attribute, and the interior points are compromises between the endpoints. Here the interior point is a forecaster for a horizon neither endpoint was trained for, and the loss has an interior minimum whose position tracks that horizon. We agree that a coupled-model or assimilation demonstration is the next step (Sec. 7). However, showing that the effect is real already required the tests that inflated the appendix: seeds, seasons, spectra, other interpolation schemes, the 1–3 h gap, two further architectures with controls.

Q1 (W1): "Why only the 3–6 h segment with two interior targets? Can the authors test a segment such as 6–12 h?"

Thank you for pointing out the 6–12 h example. The main reason we did not run it is that we had overlooked that the 12-hour Aurora checkpoint is publicly available, while pretraining such a model ourselves was out of reach. We have now run it, once again with a control (Sec. 4 "A second segment: 6–12 h", Fig. 1, Appendix I).

We fine-tuned a 6h model from Microsoft's released 12-hour checkpoint with the paper's recipe and interpolated between it (λ = 0) and the 12h checkpoint (λ = 1) for T* = 6, 7, 8, 9, 10, 12 h, every model receiving the input pair (t − T*, t) and the conditioning T*, on 64 initializations of 2022 per λ. The control interpolates Microsoft's released, independently trained 6h checkpoint with the same 12h checkpoint. The two 6h models have the same Z500 skill at 6 h (19.1 ± 3.2 vs 18.8 ± 3.9), so the two arms differ in lineage only.

The results tell the very same story as the Pangu-Weather experiment. With our fine-tune the valley appears at every interior horizon: on Z500 the minimum lies 22 % (7 h, λ* = 0.2), 29 % (8 h, 0.4), 39 % (9 h, 0.5) and 33 % (10 h, 0.7) below the better endpoint, and below both endpoints on 64 of 64 initializations at every horizon. λ* is within one grid step (0.1) of the nominal (T* − 6)/6 at all six horizons and never below it (Table 28). The independently trained models do not share a basin, and their mixture is not viable: at every horizon its midpoint is 1.7–2.6× worse than persistence (Table 29). The relative weight distance of the two pairs is 0.02 versus 1.3.

Two things follow. The two anchors are equally skilful, so it is the lineage that decides. And our limitation "connectivity decays with the lead-time gap" was wrong as stated: this segment is twice the 3–6 h span in hours and interpolates, whereas the 1–3 h gap fails because the 1h model left the basin (CKA ≤ 0.18). The Sec. 8 paragraph is now "Connectivity is local: a shared basin, and only the segment".

On what the numbers mean: the best interpolant at 7–10 h is 0.18–0.26× persistence and 22–39 % better than either endpoint asked for that horizon, but 1.6–4.3× the endpoints' error at their own horizons (about 1.4–2× on the 3–6 h segment). So we read this segment as evidence for the geometry and make no skill claim for it. Part of the level comes from the endpoints: the public 12h checkpoint was pretrained exactly like the 6h model but for 80.5k instead of 150k steps (Bodnar et al.), and on 2022 two steps of the 6h model reach Z500 25.2 at +12 h, against 31.2 for the 12h model in one step.

Q2: "Can a criterion based on CKA, weight distance or endpoint gap predict interpolation success before evaluating the intermediate targets? Has it been tested across multiple pairs?"

Yes, with one qualification. Appendix J (Table 30) collects all seven pairs of the paper with their shared initialization, weight distance, cosine, CKA where measured, and the outcome. Every shared-initialization pair gives the valley except the 1h–3h pair, whose 1h endpoint left the basin, and every one whose distance we measured sits at a relative distance ≤ 0.04. The two independently trained pairs sit at 1.2–1.3 and give the barrier. The endpoint gap in hours does not predict: 6–12 h works and 1–3 h degrades. CKA was our indicator in the submission, and it grades quality within a lineage (the 1h pairs, ≤ 0.18, give the degraded case). In practice a new pair is settled by its lineage and distance before any forecast. Having said that, for reproducing the recipe with any other model it is easy to confirm with three forward passes: the two anchors and the midpoint. That should be enough. The qualification is the sample: seven pairs (four positive with a measured valley, one degraded and two negative) is what we have, and whether permutation alignment rescues an independently trained pair is untested. Sec. 8 states the criterion with these limits.

Q3: "Are the λ values calibrated only on validation data, and do they transfer unchanged to the held-out test year? Do different variables prefer different λ?"

This is a really good question, and the submission should have been explicit about it. Actually, no calibration is performed for the headline results: Table 1 and every Aurora and Pangu-Weather forecast table use the nominal λ = 1/3 and 2/3. The λ-sweeps are diagnostics of the loss surface on a 64-initialization subset of the test year; on Aurora they report λ* = 0.30 and 0.60 and we never use them. There is one exception, and its caption now says so: the GraphCast forecasts (Table 25) were run at the sweep optima 0.30 and 0.60, read off that subset of the test year. Sec. 3 now states this.

On transfer, the nominal map is tested out of sample twice. On the second Aurora segment, a different checkpoint pair, the nominal (T* − 6)/6 is within one grid step of λ* at all six horizons (Table 28). On Pangu-Weather, a different architecture without any lead-time input, the competitive 3h anchor gives λ* = 0.35 and 0.70 against the nominal 1/3 and 2/3 (Table 21). The submission's light-budget Pangu-Weather anchor (Z500 22.2 at 3 h) had its optima at 0.50 and 0.80, and on that basis we wrote that the map is pair-specific; with an anchor trained to the official model's skill the optima fall on the nominal values, and we have withdrawn that statement (Sec. 4.1, Sec. 8). The cost of not calibrating is small: on the ten interior horizons we swept, using the nominal λ instead of the swept optimum costs at most 6.4 % in Z500 (Appendix J).

On per-variable λ: only mildly, with two exceptions. We checked it in a pilot on the 6–12 h pair that is not in the paper (eight initializations from 06 UTC bases, all 69 output variables, λ on a 0.1 grid). Every variable has an interior optimum at each of 7, 8, 9 and 10 h, and 61 to 69 of the 69 optima lie within one grid step of Z500's. The exceptions are the stratospheric levels (50–150 hPa) and, at 10 h, T2m, whose optimum is lower than Z500's at every horizon, toward the 6h model. The nominal λ, rounded to the grid, costs at most 15 % against each variable's own optimum for 62 to 69 of the 69 variables; for T2m it costs under 3 % at 7–9 h and 55 % at 10 h (one base hour, so the diurnal cycle is not sampled). On the 3–6 h segment, at the nominal λ, LWI beats every grid-space interpolant on T2m, T850, U10 and V10 at both horizons, with all eight paired-bootstrap intervals excluding zero (Table 1, Appendix D), so the nominal λ is not tuned to geopotential at the expense of the near-surface variables. The practical choice is one λ per horizon, checked on the variables that matter downstream.

Q4 (W2): practical value; "what concrete deployment setting makes LWI preferable after accounting for the cost of fine-tuning the 3h endpoint and calibrating λ?"

This makes perfect sense, and the manuscript now states it (Sec. 7). We concede the point-error gap as the paper states it: native 4h/5h models lead LWI by 7–33 %, and the 1h+3h chain leads on Z500 by 4.8 and 5.4 m²/s² (Table 1). Once again, we wanted to show LWI first as an unexpected finding, mostly from the spectral perspective. The accounting is as follows.

First of all, the developer's own route is matched: on Pangu-Weather, one fine-tune of the public 6h model and one forward pass are on par at 4 and 5 h with the chain of two independently trained official models (Q5 below; Table 22).

Training: one fine-tune next to a public checkpoint (the 3h endpoint at about 2.4 % of pretraining) serves every horizon between them; on the 6–12 h segment one fine-tuned model serves four interior horizons that would otherwise be four native fine-tunes. Calibration: none is needed, since using the nominal λ instead of the swept optimum costs at most 6.4 % in Z500 on the ten interior horizons we swept (Appendix J; Q3). Inference: one forward pass against two or three for the chain, and no hand-off between models that were never trained to compose. Spectrum: LWI preserves the ERA5 spectrum where chains and grid-space interpolants blur (Fig. 3), and for a forcing or assimilation user the variance of the forcing is the quantity that matters.

Thus, the setting (Sec. 7) is a coupled system or a sub-6 h assimilation cycle that needs the atmosphere at several cadences not divisible into 6 h, where a forecaster must exist at every needed horizon, spectral fidelity matters more than the last percent of Z500 RMSE, and a model per horizon is not affordable.

In fact, part of the geopotential gap is removable. On a 256-initialization diagnostic sample of 2022, most of LWI's Z500/MSLP excess over the spline at 4 h is a zonally uniform offset induced by conditioning on an untrained lead-time value (Sec. 5 f, Table 32). Interpolating the conditioning with the same λ as the weights, with no new parameter, lowers LWI's Z500 error by 26 % (00/12 UTC bases) and 13 % (06/18 UTC), its MSLP error by 26 % and 18 %, and its T850 and U10 errors by 8–15 % (Table 31). The chain and the native models were not run on that sample, so we do not claim that this closes the gap to them.

Q5: "Why are the 1h+3h and 1h+1h+3h chains of the official Pangu checkpoints not evaluated at 4 h and 5 h?"

On Aurora, exactly these chains are in Table 1 (1h→3h at 4 h, 1h→1h→3h at 5 h): the chain wins Z500 and loses the near-surface variables (V10 at 5 h is a tie). On Pangu-Weather the question is fair and exposed a real weakness of the submission: our 3h anchor (Z500 22.2 at 3 h against 13.1 for the official model) could not be compared with the official chains on equal terms. Thus, we fine-tuned a new 3h anchor from the official 6h weights to reach the official model's skill (Z500 13.8 at 3 h against 13.14) and repeated the sweep: the valleys are 53 % (4 h) and 38.5 % (5 h) below the better endpoint, ε(λ) < 0 at every interior λ, and λ* = 0.35 and 0.70 against the nominal 1/3 and 2/3 (Table 21, Fig. 2b). Then we benchmarked the interpolants, at the nominal and uncalibrated λ, against the official chains on the same 256 initializations (RMSE, Table 22):

Z500T850U10V10T2m
4 h, LWI27.140.4460.4920.4980.620
4 h, chain 1h→3h26.360.4520.5190.5220.622
5 h, LWI28.010.4800.5530.5650.637
5 h, chain 1h→1h→3h27.460.4890.5650.5740.645

LWI is 3 % and 2 % behind on Z500, ahead on T850, U10 and V10, and level or ahead on T2m, with one fine-tune of the public 6h model and one forward pass against two independently trained official models and two or three passes. The ranking is the one Aurora shows in Table 1, with a far smaller geopotential gap (3 % and 2 % here, against 23 % and 26 % on Aurora). This is consistent with the conditioning offset of Sec. 5 f, which Pangu-Weather cannot have: it takes a single atmospheric state and has no lead-time input, which is why it remains the cleanest mechanism test in the paper. Sec. 4.1 and Sec. 8 are updated accordingly.

The authors thank the reviewer ukia for the questions that made the findings clearer and substantially improved the manuscript.

Reviewer_g5LEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper is about temporal superresolution of neural weather models (which use 6 hours time step). The authors train 1 and 3 hour models, and then do Linear Weight interpolation to get intermediate predictions between 3 and 6 hours, and it fails when we go to 1 hours, and more advanced weight interpolation does not work.

Critical strengths

The idea of using weight interpolation in weather models for time interpolation can be quite useful, and it requires not significant retraining. However, it fails for some cases. Also, it is interesting that it beats the intepolation in time with cubic splines, so the geometry in the weights is interesting.

Critical weaknesses

I think this is really a minor issue: yes, we might need a model that predicts the field at intermediate times; yes, linear weight interpolation is an interesting concept, but there could be numerous ways of dealing with those problems. Some baselines are tested (like cubic spline interpolation), but some are not.

Questions for authors
  1. What if we add a timestep conditioning to the model and train it?
  2. What to do for the cases when simple weight interpolation does not work (for smaller time steps)?
2 comments

The authors thank the reviewer g5LE. Both questions point at routes to an intermediate horizon that the submission had not measured, and we have now measured them.

Q1: "What if we add a time-step conditioning to the model and train it?"

Aurora is already conditioned on the lead time, and the conditioning alone does not give the interior: asking the 3h model for 4 h and 5 h through its conditioning gives Z500 RMSE 48.7 and 97.0, against 23.7 and 23.9 for LWI on the same initializations (Sec. 5 d). Pangu-Weather has no lead-time input at all and shows the same valley (Sec. 4.1).

Having said that, the model you describe exists, and we compared with it. Aurora 1.5 fine-tunes Aurora with hourly targets and a randomized input-to-target gap, so that one network issues every hour of a 6 h step, at the price of a full multi-horizon fine-tune. We evaluated it against LWI between 3 and 6 h on 2024, outside its training data (Appendix M, Table 34). Aurora 1.5 leads LWI in the interior on Z500 (15.2 vs 25.7 at 4 h), MSLP and T2m, and narrowly on T850. The two dedicated endpoints beat it at their own horizons on every variable except Z500 at 6 h (at 3 h: Z500 13.5 vs 15.7), and LWI leads on the 10 m winds at every lead. So the two answer different questions: Aurora 1.5 is a forecaster trained for the sub-steps, whereas LWI asks what two existing checkpoints already encode, at no training cost for the interior.

Conditioning helps LWI as well. Interpolating the conditioning vector with the same λ as the weights, with no new parameter, removes most of the offset LWI inherits from asking for the untrained value 4 h: on a 256-initialization diagnostic sample of 2022 its Z500 and MSLP errors fall by 26 % (00/12 UTC bases; 13 % and 18 % from 06/18 UTC) and its T850 and U10 errors by 8–15 % (Table 31). Thus, conditioning and weight interpolation are complementary.

Weakness: "a minor issue; numerous ways; some baselines not tested"

On the problem: ocean, sea-ice, hydrology and air-quality models and sub-6 h assimilation cycles need forcing at cadences unrelated to 6 h. A coupled demonstration is the next step and is not in this paper.

On the finding: from our perspective the main message is that it is not at all obvious that mixing the weights of two networks of the very same architecture gives a meaningful model. Here the loss has an interior minimum whose position tracks the target horizon within 0.07 of the nominal map. We now see it on two Aurora segments (3–6 h and 6–12 h), on Pangu-Weather and GraphCast, and for three seeds, and we do not see it between independently trained checkpoints of Pangu-Weather or of Aurora. This geometry is the claim of the paper, and its controls are what inflated the appendix.

On baselines, the paper tests four training-free grid-space baselines, autoregressive chains of the 1h and 3h models, natively fine-tuned 2h, 4h and 5h models as the upper reference, SLERP, Fisher-weighted and subset interpolation, and now a conditioned multi-horizon model, Aurora 1.5. Learned temporal interpolators (now in Sec. 2) cost a forecaster-sized training run, which is the cost class of our native 4h/5h references. The spectral claim is now scoped to training-free baselines, and Sec. 8 names the learned interpolator as the missing comparison.

Q2: "What to do for the cases when simple weight interpolation does not work (for smaller time steps)?"

First, such cases can be predicted before any forecast. Appendix J (Table 30) collects the seven checkpoint pairs of the paper. Every pair that shares an initialization shows the valley, except the 1h–3h pair, and every shared-initialization pair whose distance we measured sits at a relative weight distance ≤ 0.04. The two independently trained pairs (official Pangu-Weather 3h/6h, official Aurora 6h/12h) sit at 1.2–1.3 and both show the barrier. For a new pair, three forward passes (both anchors and the midpoint) confirm it.

Second, the small-step failure is caused by a basin change; the step being short is not the reason. The 1h adaptation left the basin under the same recipe (CKA ≤ 0.18 against both other checkpoints), which is why 1h–3h interpolation reaches only 20.7 on Z500 against 12.1 for the 1h+1h chain and 11.7 for a native 2h model (Appendix H). The 6–12 h segment is twice as long in hours and interpolates at all four interior horizons (Sec. 4, Fig. 1). Our limitation "connectivity decays with the lead-time gap" was wrong as stated, and the paragraph is rewritten (Sec. 8).

Third, the remedy is a nearer anchor: the native 2h model of Appendix H is such an anchor and a useful forecaster on its own, and the two shorter segments it creates (1–2 h, 2–3 h) can be screened with the same pair statistics. Note also that even on the failing 1h–3h segment the interpolant is a working forecaster (1.7× the chain); the catastrophic failures are the independently trained pairs, and the criterion rules those out in advance.

Soundness. The review rates soundness 1 but does not say which claim is of concern, and we would like to address it directly. The controls are: three independent fine-tuning seeds with identical results; two negative controls (independently trained checkpoints of Pangu-Weather, and now of Aurora) against four positive pairs on three architectures; every 6-hourly initialization of a held-out year (N = 1460) with a paired bootstrap on every headline comparison; and headline λ fixed to the nominal 1/3 and 2/3 without calibration on any data (Sec. 3). If a specific claim appears unsupported to you, please name it and we will answer with the number or the experiment.

Presentation. We shortened the sweep-protocol paragraph (Sec. 4) and added to Sec. 3 that the headline λ is never calibrated and which sample each new experiment uses. If particular passages cost you effort, we would be glad to know which.

Comments on the paper

1 comment
Area_Chair_iV8sOpenReview ↗

Please Respond to the Authors' Rebuttal

Dear Reviewers,

As the discussion period is nearing its end, please review the authors' responses and provide your feedback, particularly on whether your concerns have been adequately addressed.

Thank you for your time and effort!

Best regards,
AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.