acceptodds
Under review as a conference paper at ICLR 2027

Physics-guided Expert Selection for Weather Forecasting

Abstract

Heterogeneous AI weather models excel under different atmospheric states and locations, creating complementary strengths that forecast fusion is expected to exploit. Yet conventional fusion blends expert predictions and may dilute the locally better forecast. To better exploit this complementarity, we develop Physics-guided Expert Selection (PES), a lightweight plug-and-play layer for spatial expert selection, using three physically interpretable diagnostics: advection, expert disagreement, and their joint magnitude. These diagnostics reveal where expert selection is beneficial, turning black-box routing into gray-box selection. PES outperforms the task-wise stronger constituent expert across all 40 tasks, reducing RMSE by up to 6.3% with only 110K task-specific routing parameters and about 50 ms per variable-lead field. Across two additional heterogeneous expert pairs, PES again improves upon the task-wise stronger constituent expert across all 40 tasks, showing that the selection framework transfers across different expert combinations.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI weather forecasting, MoE, Physics-guided machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Md3SOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper discusses how improvements to heterogeneous AI weather forecaster fusion can be made in the form of leveraging a physics-guided expert selection (PES) method to provide spatial expert selection. PES is a lightweight layer adding only 110k additional parameters and 50ms delay per forecast, but reduces RMSE across all evaluation tests.

Critical strengths
  • Important, timely problem, especially as the need to predict extreme weather events quickly and accurately increases. The work nicely showcases that preferring experts globally does not capture the complementary strengths of heterogeneous models and that adaptive preferences deliver improvements.

  • Impressively positive results on the large scale, and a convincingly broad evaluation and ablation that demonstrates the benefit of the method.

Critical weaknesses

The main issue and danger I find is the lack of smoothing in the expert selection mechanism and how that might affect the evolution of extreme events. Looking at figure 4 specifically, subfigure b suggests that the weight of individual locations can vary significantly even for adjacent data points, and thus forcing the final values to potentially have unphysical discontinuities. I am worried on the impact this can have especially on the prediction of extreme events. In detail, looking at subfig a which describes the path of a typhoon, the PES solution, even though it comes closest to reality in absolute metrics, shows (what seems to this reviewer) unphysical behavior in the drastic way the direction of the typhoon changes towards the end of the trajectory. The study only focuses on one pairwise expert selection: Pangu-Weather and NeuralGCM - while encouraging, this is not sufficient to suggest generality of PES as an approach itself.

Questions for authors
  • Please comment on the impact the lack of smoothing might have on the results
  • Could you evaluate PES' fitness in forecasting extreme weather events, and consider adding more downstream case studies to the paper?
  • Please comment on why Pangu-Weather and NeuralGCM were selected, and why only this pair was studied. If a different pair of forecasting models are selected, how is the behavior expected to differ. How easy/difficult would it be to adapt PES to work with a different pair, or a wider range of models (let's say select out of 3 or 4 at once)
1 comment
AuthorsOpenReview ↗

Response to reviewer Md3S

We thank the reviewer. The added experiments improve our evaluation of spatial routing and expert combinations. We test weight smoothing, examine subtropical-high boundary evolution, and evaluate additional expert pairs and three-expert routing. These comparisons address spatial variation, case coverage, and expert choice, while preserving the two-expert setting for mechanism analysis.

Responses to Questions

Q1. How does spatial smoothing affect the forecasts?

We smoothed the predicted weights and evaluated the original top-10% routing-gradient regions:

Smoothing kernelMean RMSE increase
3×3 averaging kernel0.287%
5×5 averaging kernel0.371%
7×7 averaging kernel0.402%

Global RMSE also increases by 0.071%, 0.106%, and 0.123%, respectively. Direct smoothing can remove useful local preferences, indicating a trade-off between smoothness and selection flexibility.

Q2. Can additional weather events be evaluated beyond the typhoon case?

We will add subtropical-high boundary evolution at a fixed 120-h lead, evaluated by boundary-latitude MAE:

MethodBoundary-latitude MAE
Pangu0.663°
NeuralGCM1.096°
IFS0.871°
PES0.405°

PES reduces error by 38.8% relative to the best individual forecast .

Typhoon centers are diagnosed as local Z850 minima on the 1.5° grid at 12-h intervals. Grid-point switching can produce zigzag segments; we will clarify this spatial-resolution and center-identification limitation.

Q3. Why choose Pangu and NeuralGCM? Can the framework accommodate other or additional experts?

Pangu and NeuralGCM offer different modeling approaches and complementary errors. Appendix E evaluates GraphCast–NeuralGCM and Pangu–GraphCast: both outperform their stronger expert on all 40 tasks, with mean RMSE reductions of 3.53% and 2.62%.

Additional Pangu–IFS, NeuralGCM–IFS, and GraphCast–IFS experiments reduce RMSE by 12.35%, 6.40%, and 8.50% on Z500/120 h and improve ACC, supporting numerical–AI combinations.

We also trained joint Pangu–NeuralGCM–GraphCast routing. On Z500/120 h, RMSE decreases from 256.533 for matched-budget two-expert PES to 252.951 (1.40%), while ACC increases from 0.949012 to 0.950341. The improvement is modest: three-expert Global static achieves RMSE 253.033 and ACC 0.950463, leaving a 0.032% RMSE gain from adaptive routing.

The main paper uses two experts to simplify weight and expert-difference visualization, mechanism evaluation, and case analysis. The additional experiment demonstrates joint three-expert routing on this task.

Responses to Weaknesses

Spatial smoothness, case coverage, and expert choice are addressed in Q1–Q3 above.

Reviewer_5arUOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PES for the spatially adaptive fusion of frozen weather forecasting models. The method takes meteorological fields at the initialization time, horizontal advection, discrepancies between expert forecasts, and the combined magnitude of these factors as inputs. A lightweight CNN–Transformer network is then used to predict grid-wise weights, which are applied to generate a final forecast via a convex combination. The authors evaluated the method across tasks involving 40 variables, various pressure levels, and different forecast lead times, using data from 2020. The evaluation also extends to three different sets of expert combinations. The results demonstrate that PES consistently outperforms the best-performing individual expert in every task. The paper further provides analyses on static fusion, input feature ablation, paired bootstrapping, and a convex combination oracle.

Critical strengths

The method offers clear practical value. For operational environments already running multiple forecasting systems, the additional training and inference costs are minimal.

The method shows gains across various variables, pressure levels, lead times, and expert combinations—a finding far more compelling than results limited to a single model combination.

The experimental design is comprehensive.

Critical weaknesses
  1. Limited innovation. The method essentially uses manually designed diagnostic features to predict spatially adaptive convex fusion weights, trained via standard MSE; the technical advancement over existing adaptive forecast fusion approaches remains unclear.
  2. Inconsistency between objectives and narrative. A reduction in fusion error may simply result from the offsetting of errors among experts, rather than the model learning to select more reliable experts. Neither the spatial structure of the weights nor their deviation from 0.5 provides sufficient evidence to support such a claim.
  3. Insufficient attribution of primary gains. For Z500 at a 120-hour lead time, static fusion already achieves a 4.282% reduction in RMSE, while the full PES model achieves 4.424%. The improvement over individual experts is primarily driven by the baseline fusion component, which is insufficient to substantiate the central role of the physics-guided mechanism.
  4. Inadequate validation of generalization. Splitting training and test sets using adjacent dates within the same year makes it difficult to assess performance for future deployment; furthermore, the bootstrap procedure at the initialization level fails to adequately account for the temporal correlation inherent in weather processes.
Questions for authors
  1. Do the learned weights actually correspond to experts with lower error rates? Does performance still improve after applying "hard selection" (choosing only the single best expert)?
  2. Is it possible to disentangle the contributions of "correct expert selection" versus "error cancellation" to the overall performance gains?
3 comments
AuthorsOpenReview ↗

Response to reviewer 5arU

Response to Reviewer 5arU

We thank the reviewer. Our additional analyses improve the interpretation of PES through soft–hard selection comparisons, error accounting, and month-cluster resampling. We distinguish total combination gains from incremental routing gains and clarify the role of physical diagnostics. These results address the mechanism and statistical concerns within the evaluated setting.

Responses to Questions

Q1. Do higher weights correspond to more accurate experts? Would hard selection perform better?

Figure 3(a–c) links stronger advection to greater relative accuracy of NeuralGCM and corresponding routing preferences. Error-gap-weighted threshold-selection accuracy averages 61.04%.

MethodMean RMSE improvement
Soft PES3.523%
Hard selection with a threshold of 0.50.326%
Probabilistic selection using the routing weights−2.258%

Probabilistic selection uses the square root of analytically expected MSE. Continuous preference and combination jointly support performance; the weights neither guarantee correct local hard selection nor represent calibrated correctness probabilities.

Q2. Do the gains mainly come from error cancellation rather than expert selection?

We partition net MSE reduction at the locally better expert’s squared-error level into improvement up to that level and additional improvement beyond both experts.

Across 40 tasks, the first component accounts for 79.98% of normalized pooled improvement. It proves that gains mainly come from expert selection rather than error cancellation.

Responses to Weaknesses

W1. Limited innovation

The novelty of PES lies not in the CNN–Transformer, MSE objective, or convex combination itself, but in formulating heterogeneous weather-model fusion as weather-state-dependent expert preference learning. Using forecast-time physical diagnostics, PES not only produces spatially adaptive weights but also enables analysis of the relationships among atmospheric conditions, relative expert accuracy, and routing behavior. Figure 3 shows consistent trends in advection intensity, NeuralGCM's relative advantage, and routing preference. This diagnostically interpretable, state-dependent selection mechanism, rather than new network components, is our methodological contribution.

W2. Inconsistency between objectives and narrative

We agree that spatial weight variation or deviation from 0.5 alone does not establish that the model has learned to select more reliable experts. We therefore further examine whether routing preferences correspond to actual expert strengths. Figure 3(a–c) shows consistent trends between routing preference and relative forecast accuracy across advection regimes. Error-gap-weighted selection accuracy averages 61.04% across 40 tasks. This relatively modest accuracy also indicates substantial room for improving expert selection. Future work will focus on increasing selection accuracy and moving closer to the theoretical upper bound of the selection framework.

W3. Attribution of performance gains

We acknowledge that conventional fusion accounts for a substantial portion of the overall improvement. On Z500/120h, Global static, Blind, and Full PES achieve RMSE reductions of 4.282%, 4.278%, and 4.424%, respectively. However, Global static is a post-hoc result obtained using ground-truth information, and is therefore not directly comparable to the forecast-time predictions of PES.

We further partition net MSE improvement at the locally better expert's squared-error level. Across 40 tasks, the first component accounts for 79.98% of normalized pooled improvement. This characterizes the composition of error reduction but does not independently establish whether the gains arise from correct selection rather than error cancellation.

We will therefore more clearly distinguish conventional fusion gains from the incremental contribution of state-dependent routing, emphasizing the value of PES as a lightweight adaptive combination framework with interpretable relationships between atmospheric conditions and expert preferences.

W4. Temporal dependence and generalization

We added 20,000 paired calendar-month bootstrap replicates, retaining all evaluated initializations within each sampled month to account for within-month temporal dependence.

All 40 tasks retain positive per-task 95% confidence intervals for RMSE reduction and ACC gain. These results support robustness within the evaluated 2020 period, but do not establish cross-year generalization or account for all training-seed uncertainty. We will explicitly clarify this scope in the manuscript.

Thank you for the supplementary analysis. I still have a few questions:

  1. Q1 / W2: Figure 3 appears to show only the response relationship, rather than providing evidence that the model has successfully implemented the correct "local expert" selection. For instance, stronger advection typically occurs in mid-to-high latitudes—regions where most models tend to perform poorly. This could lead to increased errors for both Pangu and NeuralGCM, with Pangu’s performance potentially deteriorating more rapidly. Given this, how does the widening performance gap in favor of NeuralGCM actually demonstrate that the PES mechanism is correctly switching between experts? This might need to be substantiated by analyzing the RMSE response.

  2. Furthermore, regarding the 0.326% marginal improvement attributed to hard-selection: if this mechanism effectively selects the model with the lower error at every individual grid point, one would theoretically expect a more significant gain—unless the two models are nearly identical in performance. This warrants further clarification, perhaps by presenting a few representative cases where the models exhibit notable differences.

  3. Q2: The interpretation of 79.98% still puzzles me. Consider two equally weighted points with truth (0, 0), forecasts A=(2, 1) and B=(-1, -3). The MSEs of the best fixed expert, pointwise optimal selection, and fixed 50/50 averaging are 2.5, 1, and 0.625. Under the described split, the first component accounts for (2.5−1)/(2.5−0.625)=80%, although no adaptive selection occurs. How does this distinguish averaging from correct routing? Q2 says the decomposition proves that selection dominates, while W3 acknowledges that it cannot. Could you clarify?

  4. W3: Appendix F.1 states that the static coefficient is fitted on training data and fixed during testing. Why is the 4.282% result described as post-hoc and verification-informed? Is this a different baseline? If it is training-fitted, why would comparison with PES’s 4.424% be inappropriate?

  5. W4: The month-cluster bootstrap is helpful, and I appreciate the clarification about cross-year generalization. Under this evaluation, are PES’s incremental gains over static fusion and Blind also significant, rather than only its gains over individual experts?

AuthorsOpenReview ↗

Further reply to reviewer 5arU

Further Response to Reviewer 5arU

1. Does Figure 3 establish correct local expert selection? (Q1 / W2)

We agree that Figure 3 alone does not establish correct local selection. We therefore added a conditionally stratified diagnostic on test dates strictly separated from checkpoint selection: for Z500/120h, we retrained matched Full and Blind models across three prespecified paired seeds under identical settings (with diagnostic channels zeroed in Blind). Within the same absolute-latitude band (0–30°, 30–60°, 60–90°) and within-band initial-∣advection∣|\text{advection}| quintile, we computed the area-weighted difference (in percentage points) in mean NeuralGCM routing weights between grid points where NeuralGCM actually achieves lower error and those where Pangu actually achieves lower error:

SeedFull weight difference (points)Blind weight difference (points)95% interval for Full − Blind (points)Positive Full strata
202607171.580.42[0.67, 1.70]15/15
202607182.220.40[1.11, 2.63]15/15
202607194.160.33[2.61, 5.29]15/15

All 15 stratified weight differences across all three seeds are positive and substantially exceed Blind, indicating that after controlling for latitude and advection magnitude, the model retains a correctly directed local preference association.

2. Does the ~0.326% hard-selection gain imply nearly identical expert performance?

We partitioned each task into five area-weighted quintiles based on the absolute difference between the two experts' squared errors. In the top 20% highest-gap area, error-gap-weighted accuracy for thresholded PES hard selection is 62.83% (versus 62.28% for always selecting the globally stronger expert of the task), and this quintile contributes roughly 63.7% of the mean task-normalized net hard-selection MSE gain. Thus, gains do not originate from regions where the experts perform similarly.

Across all 40 tasks, PES achieves a mean error-gap-weighted accuracy of 61.04% (versus 60.38% for always selecting the globally stronger expert). Whereas an ideal pointwise oracle using ground-truth errors reduces RMSE by 20.60% on average, current hard selection achieves 0.325%. Substantial complementarity therefore exists between the experts, but current hard thresholding captures only a small fraction of it; the small gain is not due to minimal differences between the experts.

Following a prespecified rule, we selected the initializations with the largest whole-case normalized MSE gap for each winning expert (from the 40-task production checkpoints; RMSE in m2 s−2\text{m}^2\,\text{s}^{-2}):

Task / initializationBetter expertPanguNeuralGCMEqualHard PESSoft PES
Z500_L120 / 2020-02-29T00:00NeuralGCM417.112261.505317.266270.111295.511
Z500_L72 / 2020-05-31T12:00Pangu116.861144.010120.092144.206130.493

Hard selection succeeds when NeuralGCM has a large advantage, but fails when Pangu is substantially better because the router remains biased toward NeuralGCM. We openly acknowledge this limitation.

3. Was the static coefficient fitted on training data or post hoc? (W3)

We apologize for the confusion. The values reported in the original manuscript are accurate; because the coefficients fitted on training data and post hoc are nearly identical (differing by less than 0.01%), a transcription error occurred in our previous response.

4. Are gains over static fusion and Blind significant? (W4)

We retrained strictly matched Full and Blind models across all 40 tasks (seed 20260717, identical architecture, six-epoch budget, and strict separation of training, validation, and test dates), and added two additional paired seeds (20260718 and 20260719) for Z500/120h (84 new routers in total).

Using 20,000 paired month-block bootstrap draws shared across all tasks, the 40-task mean relative RMSE reductions are:

BaselineMean RMSE reduction across 40 tasksCommon-month 95% intervalTasks improved
Blind0.218%[0.176%, 0.257%]38/40
Fixed0.249%[0.217%, 0.284%]37/40
FixedSpatial0.286%[0.255%, 0.315%]40/40
Equal1.236%[1.008%, 1.462%]40/40

All four 95% intervals lie strictly above zero, demonstrating a consistent, albeit modest, positive incremental gain when aggregated across the 40-task suite.

Reviewer_CijsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce a method for weather state dependent expert selection which selects between AI models.

Critical strengths
  • the idea is new or at least I am not aware of others doing the same.
  • there seems to be a consistent improvement in Forecast skill (Fig. 1)
Critical weaknesses
  • joint indicative difficulty indicator is barely explained.
  • as a result I had a difficulty understanding the results and their significance.
  • the router is mainly modeled to reduce RMSE so it will tend to pick the blurriest solution everywhere. Most of the experts already behave like ensemble means. Isn't this an issue? This is for instance relevant for the results in Table 2. I wonder how meaningful these results are as at the limit of predictability this only selects for blurriness.
  • results are a bit weak as they are mainly a case study and RMSE improvement.

Some missing references:

Questions for authors
  • does the method yield any insights from expert selection?
  • why only RMSE minimization
  • what about probabilistic forecasts and ensemble member prediction. would this work better?
  • what if classical forecasting systems are incorporated?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Cijs

We thank the reviewer. We improve the interpretation and evaluation of PES by clarifying Ξ\Xi, examining storm-centered gradient errors, and testing IFS as an expert. These analyses connect routing preferences to expert skill and provide evidence beyond field RMSE. We will add the subtropical-high case and suggested reference to strengthen the revised discussion.

Responses to Questions

Q1. Does expert selection provide any insights?

Figure 3 links stronger advection to greater relative accuracy of NeuralGCM and corresponding routing preferences, consistent with its physical component. Therefore, expert selection can reveal each expert’s potential strengths and tell us under what circumstances that expert is more accurate.

Q2. Why use RMSE?

We also evaluated ACC, both of RMSE and ACC are common and standard error metrics.

Q3. Would probabilistic forecasting or ensemble-member prediction work better?

The weights are trained for deterministic combination, not calibrated sampling probabilities. Across 40 tasks, soft PES, threshold selection, and probabilistic selection yield mean RMSE improvements of 3.523%, 0.326%, and −2.258%, respectively.

Thresholding α\alpha at 0.5 or sampling an expert with probability α\alpha increases error relative to soft PES. Our data do not support evaluating a dedicated ensemble-member model, so whether it performs better remains open.

Q4. What if classical forecasting systems are incorporated?

We tested three IFS combinations on Z500 at 120 h:

Expert pairRMSE reduction
Pangu–IFS12.35%
NeuralGCM–IFS6.40%
GraphCast–IFS8.50%

Despite its higher standalone error, IFS provides complementary information,so that error reduces more. These pairs yield larger relative RMSE reductions than Pangu–NeuralGCM, each measured against its own stronger expert.

Responses to Weaknesses

W1. Explanation of the joint difficulty indicator

Ξ\Xi is a forecast-difficulty proxy combining advection intensity (A\mathcal{A}) and expert disagreement (Δ\Delta). Errors generally increase with Ξ\Xi in Figure 3(a). Appendix H confirms positive correlations with stronger-expert absolute errors in all 40 tasks. We will clarify this interpretation in the main text.

W2. Does minimizing RMSE merely select blurrier forecasts?

We agree that MSE alone does not explicitly constrain fine-scale structure. Our claim is narrower: PES does not add smoothing beyond that of the constituent experts. Figure 4(d) shows spatially adaptive allocation: the NeuralGCM weight decreases east of the storm where Pangu-Weather is locally more accurate (Fig. 4(c)); Fig. 4(e) shows the lowest storm-centered Z850 RMSE. A storm-centered gradient diagnostic on the same Haishen case gives gradient RMSE 0.00260 for PES, statistically indistinguishable from the sharper expert Pangu-Weather (0.00256; Wilcoxon p = 0.97) and 12% lower than the stronger expert NeuralGCM (0.00322; p = 0.016); the gradient-magnitude ratio to ERA5 is 0.71 for PES and 0.70 for Pangu-Weather. The RMSE gain is therefore not obtained by suppressing local gradients.

W3. Evidence beyond case studies and RMSE improvements

ACC is reported in Figure 2(b), right, and Figure 2(c), middle. We will highlight these results and add a subtropical-high case evaluated by boundary-position error.

W4. Missing reference

Thank you for the recommendation. We will cite Luo et al.’s PhyDL-NWP work.

Thank you for the response and the additional experiments. I have read it carefully, but my main concerns remain, and some of the new results reinforce them.

Blending vs. selection (Q3, Q4, W2). The new numbers suggest that PES gains come from averaging rather than from selecting the better expert. Hard thresholding keeps only 0.33% of the 3.52% improvement, so almost all of the gain needs intermediate weights. The IFS result indicates the same thing: a weaker expert giving larger gains is what one expects from averaging forecasts with less correlated errors. A convex combination of imperfectly correlated fields damps small-scale variance and lowers RMSE for that reason alone. The decisive comparison is against an equal-weight average and a single globally optimal constant weight, per variable and lead time. Without it I cannot tell what the state-dependent routing contributes.

Smoothing (W2). The gradient diagnostic covers a single storm and is not convincing as presented.

Gradient RMSE is an L2 metric that rewards smooth fields, so a lower value does not show sharper forecasts. A gradient-magnitude ratio of 0.71 means PES has about 30% weaker gradients than ERA5, the same as the smoother expert. The ratio for NeuralGCM is not reported, and it is the number that matters here. A Wilcoxon p = 0.97 is not evidence of equivalence, and the sample and its spatial dependence are not described. Power spectra of PES and each expert, aggregated over the test set at several lead times, would settle the question.

Metrics (Q2, W3). “RMSE and ACC are standard” does not answer the question. ACC is closely tied to RMSE and shares its preference for smooth forecasts, so it is not independent evidence. The promised subtropical-high analysis is another case study, and I cannot evaluate results that are not shown. If you would like to complement this with a single metric spectral divergence, Sobolev seminorm etc. are much better suited.

Probabilistic forecasting (Q3). Sampling one expert per the weights and scoring with RMSE will lose by construction. My question was whether the approach extends to ensemble or probabilistic experts evaluated with proper scores (CRPS, spread–skill), which remains unaddressed.

Insights (Q1, W1). The advection–NeuralGCM link restates Figure 3 and could be obtained by stratifying the experts’ errors by advection strength, without PES. Since the difficulty indicator contains expert disagreement, its positive correlation with error is expected; the magnitudes would be more informative than the sign.

IFS (Q4). The result is for one variable at one lead time, with no detail on the IFS configuration or the verification reference.

I appreciate the added reference (W4). Overall, the response does not change my assessment, and I think that a much cleaner evaluation is needed.

AuthorsOpenReview ↗

Further reply to Reviewer Cijs

Thank you very much for your careful review and clear guidance! It is really helpful for the research. Here is our further response:

Your comments helped us separate the gains of static averaging from state-dependent routing and replace case-study arguments with full-sample quantitative diagnostics. We have completed the requested experiments and narrowed our claims in the revised manuscript accordingly.

1. Blending vs. selection (Q3, Q4, W2)

We agree that outperforming the constituent experts alone is not enough to separate the benefit of state-dependent routing from variance damping via convex averaging. Following your suggestion, we added two direct controls across all 40 Pangu–NeuralGCM tasks:

  • vs. Equal averaging: PES reduces RMSE by 1.290% on average (40/40 tasks), with a paired common-month-block 95% confidence interval of [1.057%, 1.521%].
  • vs. Training-fitted constant weight: PES reduces RMSE by 0.303% on average (39/40 tasks), with a paired common-month-block 95% confidence interval of [0.265%, 0.342%].

This shows that static combination accounts for most of the overall error reduction, while state-dependent routing provides a consistent, modest additional gain. As a stricter retrospective reference, we also fitted an MSE-optimal constant weight per task directly on test observations (a non-deployable oracle); PES still achieves lower RMSE in 39/40 tasks, with a mean reduction of 0.297%.

We also analyzed the sources of error reduction: 79.98% lies within the error level of the locally better expert, and 20.02% represents further improvement beyond both experts (i.e., the smoothing/averaging gain).

2. Power spectra and smoothing (W2)

Our earlier remark on smoothness referred qualitatively to the visual sharpness of typhoon edges, and we have also added the subtropical-high case study to the revised manuscript. We agree that quantitative spectral metrics are needed, and we report the full-sample diagnostics below:

  • Normalized KL spectral divergence (L=79): Task medians are 0.002598 for PES, 0.002032 for NeuralGCM, 0.002866 for equal averaging, and 0.002590 for the training-fitted constant. Taskwise, PES reduces spectral divergence by 5.342% on average relative to equal averaging (31/40 tasks), while increasing it by 23.020% relative to NeuralGCM (3/40 tasks improved).

While PES does exhibit smoother spectra under this metric, we view spectral divergence as measuring small-scale energy fidelity rather than serving as the primary indicator of deterministic forecast accuracy.

3. Sobolev seminorm error (Q2, W3)

Following your suggestion, we evaluated the first-order Sobolev seminorm error across all 40 tasks:

BaselineMean RMSE reductionRMSE winsMean Sobolev reductionSobolev wins
Pangu11.730%40/4010.523%40/40
NeuralGCM3.523%40/403.921%40/40
FixedWeight0.303%39/400.109%31/40

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.