- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors introduce HurdleCast, a generative precipitation nowcasting model combining latent flow matching with a supervised mixture-of-experts (MoE) for dry, moderate, and heavy precipitation regimes. Experiments on SEVIR and MeteoNet demonstrate improved performance, particularly for extreme precipitation events.
Critical strengths
- Clear presentation and thorough evaluation. The paper is well written, clearly motivated, and easy to follow. The experimental evaluation is comprehensive, with competitive baselines and informative ablation studies.
- Simple, effective contribution. The supervised hurdle-gated MoE is intuitive and well-motivated. The ablations convincingly support its effectiveness, especially for extreme precipitation.
Critical weaknesses
- Limited forecasting horizon and high computational cost. Evaluation is restricted to 60-minute forecasts, while training takes approximately 11 days on two B200 GPUs per dataset. This raises questions about efficiency and applicability to longer horizons, especially when compared to other recent work like DiffCast or PercpCast.
- Limited probabilistic and qualitative evaluation. Beyond CRPS, ensemble calibration and diversity receive little attention. The visual improvements in Figures 4 and 5 are also difficult to appreciate, and the local SSIM maps are not particularly informative.
Questions for authors
- Could the authors compare against PercpCast and DiffCast, including training efficiency? PercpCast (3-hour forecasting) reportedly trains in 16 hours on an A100, and DiffCast (100-minutes forecasting) in under a day on an A6000. They both achieve SOTA performance in SEVIR and have public code.
- Could the authors provide additional ensemble calibration and diversity analysis, such as spread-skill plots and rank histograms?
- Could the authors clarify the local SSIM maps, report LPIPS in the main comparisons, and explain the white regions in exPreCast’s prediction in Figure 5?
- Could the authors consider an anonymized code release?
5 comments
Q2: Ensemble Calibration and Reliability
We thank the reviewer for suggesting additional probabilistic diagnostics. We have conducted the requested reliability analysis on the full SEVIR test set, using eight-member forecast ensembles.
Rank histogram. We computed a rank histogram with random jitter to handle ties arising from VIL's substantial zero inflation. The resulting histogram is U-shaped, indicating ensemble underdispersion. In particular, the lowest rank bin contains approximately 27.5% of observations, compared with 11.1% expected across nine equally likely rank bins for an eight-member ensemble.
Exceedance-probability reliability. We additionally evaluated predicted exceedance probabilities against observed event frequencies at the two highest precipitation thresholds, VIL ≥ 181 and VIL ≥ 219.
| Predicted probability | Observed frequency (VIL ≥ 181) | Observed frequency (VIL ≥ 219) |
|---|---|---|
| 0.000 | 0.001 | 0.000 |
| 0.125 | 0.105 | 0.079 |
| 0.250 | 0.177 | 0.136 |
| 0.375 | 0.239 | 0.189 |
| 0.500 | 0.301 | 0.245 |
| 0.625 | 0.368 | 0.307 |
| 0.750 | 0.447 | 0.384 |
| 0.875 | 0.553 | 0.492 |
| 1.000 | 0.769 | 0.717 |
The observed frequencies increase monotonically with predicted exceedance probabilities at both thresholds. However, predicted probabilities generally exceed observed frequencies, indicating overforecasting of extreme-event probabilities.
Empirical ensemble CRPS. To complement these calibration diagnostics, we also computed CRPS directly from the raw ensemble members, without assuming a parametric predictive distribution, using the bias-corrected estimator [4]. On SEVIR, HurdleCast's empirical CRPS is 6–7% lower than its Gaussian-fitted CRPS across lead times. It also improves upon FlowCast's empirical CRPS by 0.00046, with a 95% paired bootstrap confidence interval for HurdleCast minus FlowCast of [−0.00048, −0.00045].
These findings distinguish improvements in overall probabilistic accuracy from the remaining calibration limitations. The empirical CRPS results support HurdleCast's probabilistic advantage, while the rank histogram and reliability diagrams identify underdispersion and overforecasting of extreme-event probabilities. Ensemble size influences the interpretation of probabilistic verification diagnostics [2], and underdispersion is a recognized challenge in precipitation ensemble prediction [3].
We acknowledge these limitations and will include the additional diagnostics and their interpretation in the appendix.
Q3: LPIPS and Qualitative Evaluation
We thank the reviewer for requesting additional perceptual evaluation and clarification of the qualitative figures.
LPIPS comparisons. HurdleCast's LPIPS is already reported in Appendix A.13 through the ensemble-size analysis (LPIPS = 0.2744 for eight ensemble members on SEVIR). We provide the comparative results below and will incorporate them into the main experimental comparisons.
SEVIR LPIPS (lower is better):
| Model | LPIPS |
|---|---|
| U-Net | 0.4010 |
| EarthFormer | 0.3946 |
| SimVP | 0.3622 |
| exPreCast (deterministic) | 0.1964 |
| CasCast (N=8) | 0.2811 |
| FlowCast (N=8) | 0.2767 |
| HurdleCast (N=1) | 0.2349 |
| HurdleCast (N=8) | 0.2744 |
MeteoNet LPIPS (lower is better):
| Model | LPIPS |
|---|---|
| U-Net | 0.3102 |
| EarthFormer | 0.3081 |
| SimVP | 0.2739 |
| exPreCast (deterministic) | 0.1824 |
| FlowCast (N=8) | 0.2510 |
| HurdleCast (N=1) | 0.2164 |
| HurdleCast (N=8) | 0.2458 |
The three deterministic regression baselines (U-Net, EarthFormer, and SimVP) exhibit higher LPIPS than the generative methods on both datasets, consistent with the smoothing often associated with pixel-wise reconstruction objectives.
Among the remaining methods, exPreCast achieves the lowest LPIPS. Ensemble averaging also affects perceptual similarity: HurdleCast's single-member prediction (N=1) achieves substantially lower LPIPS than its eight-member ensemble mean, consistent with the smoothing introduced by averaging generated forecasts.
For eight-member ensembles, HurdleCast achieves LPIPS of 0.2744 on SEVIR, compared with 0.2767 for FlowCast and 0.2811 for CasCast. On MeteoNet, HurdleCast achieves 0.2458, compared with 0.2510 for FlowCast. These results place HurdleCast among the stronger probabilistic baselines in perceptual similarity, while its principal advantage remains extreme-event forecasting skill.
Local SSIM maps (Figures 4 and 5). The local SSIM maps identify spatial regions where forecast structure differs from the ground truth. Two forecasts may appear visually similar at the full-image scale while exhibiting meaningful local differences in storm boundaries, spatial texture, and precipitation-core placement. The maps are intended to highlight these discrepancies rather than serve as an additional aggregate performance metric.
We agree that their interpretation could be clearer and will expand the figure captions to explain the local structural differences they illustrate.
White regions in exPreCast's Figure 5. These regions correspond to negative dBZ values. Because dBZ is logarithmic, negative reflectivity values can represent very weak radar returns. The white regions arise because these values fall below the colormap's 0 dBZ display floor.
Unlike the other models, exPreCast's output was not clipped to the 0 dBZ display floor to remain faithful to its released implementation.
W1/Q1: Forecasting Horizon, Baselines, and Computational Cost
We thank the reviewer for their positive assessment of the paper's presentation, comprehensive evaluation, and the effectiveness of our supervised hurdle-gated MoE. We appreciate the suggestions concerning forecasting horizons, efficiency, probabilistic evaluation, and reproducibility, and address them in separate responses.
Benchmark protocol and forecasting horizon. Our evaluation compares HurdleCast against eight baselines: six ML-based approaches (U-Net, EarthFormer, SimVP, exPreCast, CasCast, and FlowCast) and two conventional nowcasting approaches (Persistence and pySTEPS). All are evaluated under the same protocol at SEVIR's native 384×384 resolution and a 60-minute forecasting horizon (Table 1), consistent with the benchmark setting established in SEVIR's introductory paper [1].
PercpCast and DiffCast evaluate different forecasting configurations:
These differences in forecasting horizons, spatial resolutions, and evaluation protocols prevent direct comparison of the published benchmark results.
Reproducibility of additional baselines. We examined both publicly released implementations and identified practical obstacles to reproducing them under our evaluation protocol:
DiffCast: Although the paper describes joint training of the diffusion component and deterministic predictor (Section 4.4), the released implementation raises
NotImplementedErrorwhen training is invoked, stating that the training process is unsupported. The available code supports inference with the pretrained 128×128 checkpoint using 5 input and 20 output frames, but not retraining under our configuration.PercpCast: Its repository includes a configuration file matching our resolution and frame counts, but our inspection indicates that this configuration is not used by the executable pipeline. The released pipeline instead uses 128×128 resolution and a 36-frame, 180-minute forecasting output.
Consequently, reproducing either method under the native SEVIR protocol requires substantial implementation changes beyond adjusting configuration parameters. We therefore prioritize comparisons with eight established baselines under identical evaluation conditions. We recognize that longer-horizon forecasting remains an important direction that our current experiments do not assess.
Training and inference efficiency. For completeness, we summarize the reported training costs of the closest generative baselines alongside our measured inference latency.
CasCast's reported training time covers only its diffusion stage, excluding the separately trained autoencoder and EarthFormer compression model. As these approaches use different hardware and training configurations, the figures provide context rather than a hardware-normalized comparison.
We additionally measured inference latency for an eight-member ensemble at batch size one under identical conditions:
HurdleCast introduces a negligible inference overhead of 0.02 s per input (approximately 0.18%) relative to FlowCast, while improving extreme-threshold forecasting skill (Table 4). CasCast achieves lower inference latency, illustrating the trade-off between computational efficiency and extreme-event forecasting performance across these approaches.
We will incorporate these comparisons and clarify the computational considerations in the revised manuscript.