- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Motivated by the high cost of diffusion-based probabilistic DLWF models, this paper proposes Hyan, an efficient ensemble weather forecasting model with state-adaptive noise injection into intermediate representations, and Hyan DE, its deep-ensemble variant. A lightweight bottleneck network produces spatially varying, state-dependent perturbations, requiring only one forward pass per ensemble member. On WeatherBench2, Hyan DE reduces CRPS by 7.92% relative to IFS ENS and outperforms ArchesWeatherGen at only 10.76% of its estimated inference cost.
Critical strengths
-
The proposed state-adaptive perturbation mechanism enables efficient ensemble forecasting without iterative diffusion sampling.
-
This paper is easy to follow.
Critical weaknesses
-
My biggest concern is the insufficient motivation and physical justification for intermediate perturbations proposed in this work. Typically, ensemble forecasting aims to characterize uncertainty in physically plausible atmospheric evolution, rather than merely introduce variability into predictions. For example, the recent GenCast model introduces Gaussian noise as part of its input because it assumes that Gaussian noise can characterize the uncertainty inherent in weather states and the noise in observed weather data. Take diffusion-based models as another example; they inject noise directly into weather data in a finer way. However, in this work, Hyan injects stochastic perturbations into intermediate latent representations without sufficiently justifying what sources of physical uncertainty these perturbations represent or whether the resulting forecasts remain physically plausible and interpretable. The authors should conduct further investigations into the physical plausibility and interpretation of these perturbations and provide a more fundamental justification for the proposed method before resubmission.
-
It remains a concern about fair computational comparisons (the key motivation of this work). The reported inference-cost advantage relies on approximate hardware equivalences based on theoretical peak computational capabilities rather than measured end-to-end performance. Several baselines also use different spatial resolutions.
-
Several design choices, including the bottleneck dimension, perturbation locations, spherical noise spectrum, and autoregressive rollout schedule, lack systematic sensitivity analysis. Their effects on forecast skill and training stability remain unclear.
-
The main CRPS improvements are reported without confidence intervals or significance tests. In particular, the 1.46% improvement over ArchesWeatherGen requires stronger statistical evidence.
-
The experiments mainly evaluate aggregate forecasting metrics, leaving the model's ability to represent rare events, such as tropical cyclone prediction. This might be a necessary empirical investigation which has been done in other DLWF models such as GenCast for justifying the effectiveness of the proposed approach.
Questions for authors
Please refer to the weakness. I am open to increasing my scores if the authors can fully address my concerns.
4 comments
W4. The main CRPS improvements are reported without confidence intervals...
We thank the reviewer for this comment. ArchesWeatherGen is a strong 1.5° baseline, and in particular its results on the WeatherBench2 leaderboard have benefited from extensive engineering optimization. In Fig. 3 we include almost all representative ERA5 variables, four surface and five pressure-level variables, nine in total. Taking the 24-hour forecast as an example, Hyan improves on the pressure-level variables (z500, mslp) by about 2.5%, on the wind variables (u850, v850, u10, v10) by close to 5.5%, and on the temperature variable t850 by 1.11%, with a decrease only on t2m (-5.07%) and q850 (-1.71%). This indicates that the proposed method does offer a measurable advantage. On the other hand, under the inference-speed evaluation protocol of ArchesWeatherGen, our inference cost is only about 10%, which shows that Hyan achieves competitive or even better results at very low inference cost. We would argue that this already supports the conclusion of the paper.
W5. The experiments mainly evaluate aggregate forecasting metrics, leaving the model's ability...
We thank the reviewer for raising this point. We agree that the original evaluation did not cover this point. We therefore select eight representative typhoons from 2020, including Goni, the strongest typhoon of that year, and compare our model against IFS ENS with 40 ensemble members. The IFS ENS results are taken from WeatherBench2 and likewise use 40 members. The statistics are summarised below.
| Lead time (h) | 24 | 48 | 72 | 96 | 120 |
|---|---|---|---|---|---|
| Ours | 60.1 ± 29.2 | 91.7 ± 29.3 | 159.9 ± 49.1 | 261.3 ± 94.0 | 379.5 ± 105.0 |
| IFS ENS | 85.9 ± 41.1 | 144.5 ± 37.6 | 227.0 ± 87.8 | 323.7 ± 180.1 | 623.3 ± 403.2 |
In addition, we use Chan-Hom as a representative case for visualisation analysis, as shown in Fig. 4 of the paper.
Reply to the authors' rebuttal
I appreciate the authors' clarifications and additional experiments. However, several concerns of mine still remain unresolved.
W1. The rebuttal does not sufficiently address the physical motivation and theoretical justification of the proposed perturbations. Comparisons with limitations of existing methods do not establish the validity of the proposed approach.
W2. The updated computational results are appreciated, but concerns regarding fair and consistent comparisons remain.
W3. Engineering design and choices are essential for bridging the gap between high-level methodological principles and practical deployment. While conceptual contributions may suffice for simpler or proof-of-concept problems, ensemble deep learning weather forecasting is a complex and highly competitive field attracting attention from both academia and industry. Systematic investigation of these choices, together with deeper technical design and theoretical justification, is necessary to establish the practical value and maturity of this work.
W4. The rebuttal does not provide statistical evidence to support the claimed improvements, particularly over strong baselines. Statistical evidence is very important to justify that the improvement achieved by a new work is really better than its previous counterparts, rather than biased experimental observations.
W5. The additional experiments are appreciated, but their limited scope does not yet establish the model's general reliability in predicting extreme events.
Overall, I believe the work requires more thorough investigation and stronger theoretical and empirical support to meet the expectations of this research community. I therefore maintain my original assessment and encourage the authors to further strengthen the paper before resubmission.
We thank the reviewer for the feedback. We would like to clarify a few points below.
W1. Our method is motivated precisely by the shortcomings of prior approaches, and its physical motivation rests on both intuition and empirical observation: perturbations should not be uniformly distributed across the global space, as prior methods assume, but should be correlated with the state. We did observe such a correlation in the autoregressive (AR) process, and the intensity-distribution visualization already present in the paper provides supporting evidence for it. In the latest revision, we have strengthened this argument in Figure 2.
W2. As explained in our earlier response, all evaluations follow the framework proposed by ArchesWeatherGen and UCast. The proposed Hyan method was pre-warmed up on a physical A100 GPU and evaluated as the average of 30 repeated measurements, which ensures that the comparison is fair and stable.
W3. As noted in our earlier response, we have conducted thorough ablations on the engineering choices underlying our core method, in particular on the source of randomness, and we compare against mainstream methods under fully identical conditions. The experimental settings were chosen so as to prioritize actual performance, so that the reported results reflect model capability under realistic conditions.
W4. In the revised manuscript we describe the CRPS performance as "competitive" because, given the substantially smaller parameter count and the significantly better inference time, an improvement of close to 1.5% is already of considerable value. For reference, the relative improvement of GenCast over ArchesWeatherGen is only about 3%. We would also like to note that training models for the WeatherBench2 leaderboard is very time-consuming. A single run consisting of training, fine-tuning, and full-year 2020 evaluation takes about a week, so evaluating 5 seeds alone would require more than a month. For this reason, multi-seed confidence intervals are rarely reported in this line of work, and we follow the same convention.
W5. Our method is not specialized for extreme weather, but aims at general probabilistic forecasting evaluation, which is why we adopt WeatherBench2 as the evaluation protocol. To our knowledge, WeatherBench2 does not prescribe any specific setting for extreme weather. In this line of work, extreme weather is typically evaluated through tropical cyclone track forecasting, and following the reviewer's suggestion we have added such an evaluation to the revision. Since tropical cyclone track forecasting is the standard proxy for extreme weather, we believe this addition provides a meaningful and representative assessment of model behavior under such conditions.
We hope these clarifications address the reviewer's concerns.
Thank you for your comment.
We respectfully disagree on this point. GenCast and DPMs share the same underlying principle, and we are not aware of evidence that denoising-based sampling leads to more physically plausible forecasts. GenCast, for instance, still shows long-term drift on many metrics.
We would also note that our method belongs to a different line of work, namely efficient ensemble forecasting (FourCastNet3 [1], FGN [2], AIFS-CRPS [3]). Prior methods do not account for the local state and rely on fixed or global perturbations, which limits how well they model the optimal perturbation direction. We therefore propose an efficient perturbation method conditioned on the latent state. Ablations show that it improves accuracy, and our visualisations show that perturbation strength correlates to some degree with humidity.
We thank the reviewer for this correction. For GenCast, ArchesWeatherGen and NeuralGCM, the average per-step, per-member inference time on the A100 is taken from the two papers cited above [4, 5]. For UCast, Hyan and Hyan DE, we instead measured the inference time directly on an A100 in FP32 using
torch.compile, with compilation and warm-up completed and the average taken over repeated runs. The total inference time per forecast is summarised below. These new results have been added to Fig. 1, and the description of the conversion has been updated accordingly.We sincerely appreciate this important suggestion. The core contribution of this paper lies in how to construct a better perturbation mechanism that makes the forecasts more accurate, rather than in the engineering choices of the backbone, and this is why we have only briefly covered these aspects. In fact, most of the architectural design is not original to this paper. The backbone follows ArchesWeatherGen [4], and the noise components follow FourCastNet3 [1], GenCast [6] and others, all of which are already in common use. Our experiments therefore focus on ablating the mechanism of noise injection and on the corresponding analyses, namely the comparison between conventional perturbation methods and the method proposed here (Sec. 6.1) and the visualisation of the perturbation distribution (Sec. 7).
[1] Bonev, Boris, et al. "FourCastNet 3: A geometric approach to probabilistic machine-learning weather forecasting at scale." arXiv preprint arXiv:2507.12144 (2025).
[2] Alet, Ferran, et al. "Skillful joint probabilistic weather forecasting from marginals." arXiv preprint arXiv:2506.10772 (2025).
[3] Lang, Simon, et al. "AIFS-CRPS: ensemble forecasting using a model trained with a loss function based on the continuous ranked probability score." npj Artificial Intelligence 2.1 (2026): 18.
[4] Couairon, Guillaume, et al. "ArchesWeatherGen: Skillful and compute-efficient probabilistic weather forecasting with machine learning." Science Advances 12.17 (2026): eadx2372.
[5] Cachay, Salva Rühling, Duncan Watson-Parris, and Rose Yu. "U-Cast: A Surprisingly Simple and Efficient Frontier Probabilistic AI Weather Forecaster." arXiv preprint arXiv:2604.09041 (2026).
[6] Price, Ilan, et al. "Gencast: Diffusion-based ensemble forecasting for medium-range weather." arXiv preprint arXiv:2312.15796 (2023).