- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper discusses how improvements to heterogeneous AI weather forecaster fusion can be made in the form of leveraging a physics-guided expert selection (PES) method to provide spatial expert selection. PES is a lightweight layer adding only 110k additional parameters and 50ms delay per forecast, but reduces RMSE across all evaluation tests.
Critical strengths
-
Important, timely problem, especially as the need to predict extreme weather events quickly and accurately increases. The work nicely showcases that preferring experts globally does not capture the complementary strengths of heterogeneous models and that adaptive preferences deliver improvements.
-
Impressively positive results on the large scale, and a convincingly broad evaluation and ablation that demonstrates the benefit of the method.
Critical weaknesses
The main issue and danger I find is the lack of smoothing in the expert selection mechanism and how that might affect the evolution of extreme events. Looking at figure 4 specifically, subfigure b suggests that the weight of individual locations can vary significantly even for adjacent data points, and thus forcing the final values to potentially have unphysical discontinuities. I am worried on the impact this can have especially on the prediction of extreme events. In detail, looking at subfig a which describes the path of a typhoon, the PES solution, even though it comes closest to reality in absolute metrics, shows (what seems to this reviewer) unphysical behavior in the drastic way the direction of the typhoon changes towards the end of the trajectory. The study only focuses on one pairwise expert selection: Pangu-Weather and NeuralGCM - while encouraging, this is not sufficient to suggest generality of PES as an approach itself.
Questions for authors
- Please comment on the impact the lack of smoothing might have on the results
- Could you evaluate PES' fitness in forecasting extreme weather events, and consider adding more downstream case studies to the paper?
- Please comment on why Pangu-Weather and NeuralGCM were selected, and why only this pair was studied. If a different pair of forecasting models are selected, how is the behavior expected to differ. How easy/difficult would it be to adapt PES to work with a different pair, or a wider range of models (let's say select out of 3 or 4 at once)
Response to reviewer Md3S
We thank the reviewer. The added experiments improve our evaluation of spatial routing and expert combinations. We test weight smoothing, examine subtropical-high boundary evolution, and evaluate additional expert pairs and three-expert routing. These comparisons address spatial variation, case coverage, and expert choice, while preserving the two-expert setting for mechanism analysis.
Responses to Questions
Q1. How does spatial smoothing affect the forecasts?
We smoothed the predicted weights and evaluated the original top-10% routing-gradient regions:
Global RMSE also increases by 0.071%, 0.106%, and 0.123%, respectively. Direct smoothing can remove useful local preferences, indicating a trade-off between smoothness and selection flexibility.
Q2. Can additional weather events be evaluated beyond the typhoon case?
We will add subtropical-high boundary evolution at a fixed 120-h lead, evaluated by boundary-latitude MAE:
PES reduces error by 38.8% relative to the best individual forecast .
Typhoon centers are diagnosed as local Z850 minima on the 1.5° grid at 12-h intervals. Grid-point switching can produce zigzag segments; we will clarify this spatial-resolution and center-identification limitation.
Q3. Why choose Pangu and NeuralGCM? Can the framework accommodate other or additional experts?
Pangu and NeuralGCM offer different modeling approaches and complementary errors. Appendix E evaluates GraphCast–NeuralGCM and Pangu–GraphCast: both outperform their stronger expert on all 40 tasks, with mean RMSE reductions of 3.53% and 2.62%.
Additional Pangu–IFS, NeuralGCM–IFS, and GraphCast–IFS experiments reduce RMSE by 12.35%, 6.40%, and 8.50% on Z500/120 h and improve ACC, supporting numerical–AI combinations.
We also trained joint Pangu–NeuralGCM–GraphCast routing. On Z500/120 h, RMSE decreases from 256.533 for matched-budget two-expert PES to 252.951 (1.40%), while ACC increases from 0.949012 to 0.950341. The improvement is modest: three-expert Global static achieves RMSE 253.033 and ACC 0.950463, leaving a 0.032% RMSE gain from adaptive routing.
The main paper uses two experts to simplify weight and expert-difference visualization, mechanism evaluation, and case analysis. The additional experiment demonstrates joint three-expert routing on this task.
Responses to Weaknesses
Spatial smoothness, case coverage, and expert choice are addressed in Q1–Q3 above.