- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors of this paper used VQ-VAE to discretize radar echoes into tokens, which were then fed into a diffusion layer to achieve probabilistic precipitation nowcasting. On SEVIR, the pooling scale mCSI (Pool16) reached 0.6348 with a 1-hour lead time. Furthermore, it can be trained on a single 32GB GPU.
Critical strengths
The advantages are:
- 128×128 radar frames are frozen and compressed into a 16×16 token grid (codebook 512) using VQ-VAE, with diffusion occurring on 256 discrete states per frame, rather than a dense, continuous latent state.
2.The diffusion path itself is controllable: a perplexed statistical construct is used to drive the erosion kernel, and combined with a precisely derived reverse rate and path-following sampler, resulting in significant benefits under low step budget.
3.Leading in pooling-scale event detection capabilities
Critical weaknesses
Weaknesses:
-
Limited dataset: Experiments were only conducted on one dataset, and the method's generalization to other radar systems/climate zones was not verified. Incidentally, CasCast at least conducted experiments on SEVIR and TST-BRAZIL.
-
Most metrics were not tested: In Table 1, 10 out of 13 methods had empty SSIM/CRPS/HSS/mCSI@60min parameters.
-
Sensitivity experiments were not conducted for longer forecast time domains such as 90 min, 2, etc., or different historical lengths or frame intervals; however, the advantage of longer lead times is greater, but I didn't see how long this prediction duration actually was.
Questions for authors
See Weaknesses
4 comments
Response to Reviewer Cpp3
Evaluation on MeteoNet
To evaluate DPPN on an additional radar dataset, we trained the tokenizer and DPPN-cas on MeteoNet, a reflectivity dataset from Météo-France. The system uses 12 observed frames to forecast the next 12 frames on a 400 × 400 grid at 5-minute intervals, for a 60-minute forecast horizon. (https://meteofrance.github.io/meteonet/english/data/rain-radar/)
On the test set, we obtain the final forecast using two ensemble-fusion strategies. The first is the conventional ensemble mean, computed by averaging the eight ensemble members pixel-wise in image space. The second is our percentile-based pixel-wise fusion described in Section 4.1. Specifically, at each pixel, we compute the median (P50) and the 60th percentile (P60) across the eight ensemble members. We select P60 when its intensity is at least 35 and P50 otherwise. CSI-M is the mean CSI over thresholds of 19, 28, 35, 40, and 47 dBZ, while CSI-47 evaluates detection specifically at 47 dBZ. For Pool4 and Pool16, non-overlapping block max pooling is applied to both the forecast and observation before computing the CSI metrics.
| Model | CSI-M Pool1 | CSI-M Pool4 | CSI-M Pool16 | CSI-47 Pool1 | CSI-47 Pool4 | CSI-47 Pool16 | CRPS ↓ |
|---|---|---|---|---|---|---|---|
| ConvLSTM [NeurIPS 15] | 0.3008 | 0.3050 | 0.3465 | 0.0982 | 0.1091 | 0.1588 | 0.0218 |
| PredRNN [NeurIPS 17] | 0.2914 | 0.3003 | 0.3402 | 0.0823 | 0.0990 | 0.1462 | 0.0214 |
| PhyDNet [CVPR 20] | 0.3120 | 0.3124 | 0.3356 | 0.1106 | 0.1157 | 0.1482 | 0.0216 |
| SimVP [CVPR 22] | 0.3017 | 0.3143 | 0.3577 | 0.0997 | 0.1134 | 0.1599 | 0.0218 |
| LDM [CVPR 22] | 0.2131 | 0.2191 | 0.2369 | 0.0359 | 0.0407 | 0.0552 | 0.0209 |
| EarthFormer [NeurIPS 22] | 0.2831 | 0.2855 | 0.3154 | 0.0787 | 0.0872 | 0.1208 | 0.0224 |
| PreDiff [NeurIPS 23] | 0.2546 | 0.2668 | 0.2935 | 0.0490 | 0.0594 | 0.0867 | 0.0197 |
| NowcastNet [Nature 23] | 0.2955 | 0.3232 | 0.3734 | 0.1236 | 0.1521 | 0.2115 | 0.0277 |
| CasCast (EarthFormer) [ICML 24] | 0.3156 | 0.3650 | 0.4420 | 0.1204 | 0.1563 | 0.2357 | 0.0180 |
| DPPN (Earthformer)-mean | 0.4176 | 0.1940 | 0.0165 | ||||
| DPPN (Earthformer)-fusion | 0.3369 | 0.3885 | 0.4678 | 0.1414 | 0.3114 | 0.0165 |
For both ensemble aggregation strategies, DPPN-Cas achieves the best performance among the compared methods in both CSI-M and CSI-47. Using the same Earthformer deterministic forecast, CasCast obtains CSI-M and CSI-47 scores of 0.4420 and 0.2357, respectively. FREUD+LSM-L [CVPR 2026] also reports results on MeteoNet, achieving an mCSI of 0.1417 and CRPS of 0.0193 under its specified data split. However, as its code and checkpoints are not available to us, we are unable to reproduce the evaluation under either our or their experimental setting for a direct comparison.
Response to Reviewer Cpp3
Missing metrics in Table 1
Thank you for pointing this out. We agree that the original Table 1 did not make the source and availability of the baseline results sufficiently clear. Our original intention was to provide a broad comparison with existing SEVIR forecasting methods in a single table. However, the availability of checkpoints and evaluation outputs differs across baselines. For several methods, including ConvLSTM and PredRNN, matching SEVIR checkpoints are not available to us. We therefore report their published CRPS and CSI results from the PreDiff benchmark (Table 2), but cannot consistently evaluate additional metrics such as SSIM, HSS, and 60-minute mCSI.
In the original Table 1, these unavailable evaluations were indicated by “–”. We agree that this notation can be misinterpreted as missing experiments rather than unavailable evaluations. PreDiff provides an official checkpoint, but reproducing the full ensemble-based evaluation is computationally expensive: one complete SEVIR test-set ensemble evaluation takes approximately 110 hours under our current setup. This computational cost also reflects one of the practical motivations for DPPN, which performs discrete diffusion on compact radar-token grids to reduce the cost of probabilistic forecasting.
To avoid ambiguity, we have reorganized the original Table 1 into two separate tables according to evaluation availability. The first table reports the metrics consistently available from the published benchmark, namely CRPS and aggregate mCSI at Pool1, Pool4, and Pool16. The second table contains methods for which suitable checkpoints and implementations are available to us, allowing a more complete and consistent evaluation including SSIM, CRPS, HSS, aggregate mCSI, and 60-minute mCSI. This separation avoids using “–” for metrics that cannot be evaluated and makes the source and scope of each comparison explicit.
Revised Table 1(a): SEVIR forecast quality using published benchmark results
| Model | Size (M) | CRPS ↓ | mCSI Pool1 ↑ | mCSI Pool4 ↑ | mCSI Pool16 ↑ |
|---|---|---|---|---|---|
| Persistence | — | 0.0526 | 0.2613 | 0.3702 | 0.4690 |
| UNet | 16.6 | 0.0353 | 0.3593 | 0.4098 | 0.4805 |
| ConvLSTM | 14.0 | 0.0332 | 0.4185 | 0.4452 | 0.5135 |
| PredRNN | 46.6 | 0.0306 | 0.4080 | 0.4497 | 0.5005 |
| PhyDNet | 13.7 | 0.0319 | 0.3940 | 0.4379 | 0.4854 |
| E3D-LSTM | 35.6 | 0.0297 | 0.4038 | 0.4492 | 0.4961 |
| Rainformer | 184.0 | 0.0357 | 0.3661 | 0.4232 | 0.4738 |
| DGMR | 71.5 | 0.0435 | 0.2675 | 0.3431 | 0.4832 |
| VideoGPT | 99.6 | 0.0381 | 0.3653 | 0.4349 | 0.5798 |
| LDM | 438.6 | 0.0280 | 0.3580 | 0.4022 | 0.5522 |
| PreDiff | 220.5 | 0.0246 | 0.4100 | 0.4624 | 0.6244 |
The first table therefore provides the broad comparison supported by the metrics available for these published baselines, without using “–” to represent metrics that cannot be consistently evaluated.
Revised Table 1(b): Complete SEVIR evaluation for models with suitable checkpoints and implementations
| Model | Size (M) | SSIM ↑ | CRPS ↓ | HSS ↑ | mCSI Pool1 ↑ | mCSI Pool4 ↑ | mCSI Pool16 ↑ | mCSI at 60 min Pool1 ↑ | mCSI at 60 min Pool4 ↑ | mCSI at 60 min Pool16 ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| Earthformer | 15.1 | 0.7632 | 0.0304 | 0.5390 | 0.4163 | 0.4309 | 0.4672 | 0.2974 | 0.3034 | 0.3270 |
| CasCast | 309.4 | 0.7587 | 0.0210 | 0.5538 | 0.4014 | 0.4885 | 0.6280 | 0.3137 | 0.3568 | 0.4554 |
| FREUD+LSM-L | 473.0 | 0.7373 | 0.0201 | 0.5276 | 0.4141 | 0.4635 | 0.5395 | 0.3003 | 0.3322 | 0.3935 |
| FlowCast | 230.4 | 0.7866 | 0.0186 | 0.5607 | 0.4417 | 0.4812 | 0.5609 | 0.3276 | 0.3572 | 0.4332 |
| DPPN-direct | 92.6 | 0.8348 | 0.0203 | 0.5351 | 0.4114 | 0.4683 | 0.6095 | 0.3165 | 0.3671 | 0.5295 |
| DPPN-Cas | 102.0 | 0.8331 | 0.0216 | 0.5466 | 0.4186 | 0.5049 | 0.6348 | 0.3288 | 0.3783 | 0.4742 |
The second table reports the complete set of metrics only for methods for which suitable checkpoints and implementations are available to us, allowing these metrics to be evaluated consistently. HSS averages the six thresholds at native resolution. In the revised manuscript, bold and underline indicate the best and second-best available results within each corresponding comparison table. We believe this restructuring makes the comparison more transparent: the absence of SSIM, HSS, or 60-minute mCSI for methods in Table 1(a) reflects evaluation availability rather than omitted experiments, while Table 1(b) provides the complete metric comparison where consistent evaluation is possible.
Response to Reviewer Cpp3
Forecast horizon and sensitivity
Thank you for pointing this out. We agree that the original manuscript did not make the forecast horizon sufficiently explicit and did not investigate sensitivity to longer forecast horizons, different observation-window lengths, or frame intervals.
The main experiments in the submitted manuscript use seven historical radar frames to predict the subsequent six frames at a 10-min interval, corresponding to a 60-min forecast horizon. To examine behavior beyond this training horizon, we have added an evaluation extending the forecasts to 120 min without retraining the model.
Specifically, the trained model first uses the seven observed frames to predict six future frames (10–60 min). We then construct a second seven-frame conditioning sequence from the last observed frame and these six predicted frames,
and apply the same trained model again to predict .
Thus, we use the seven observed frames
to predict
where and denote the observed and forecast frames at minutes relative to the forecast start. The 70–120 min forecasts are therefore obtained through an autoregressive extension of the original 60-min model, without additional training or fine-tuning. The first two forecast starts of each of the 4,053 four-hour SEVIR test events provide observations through 120 min, giving 8,106 samples for this evaluation.
| Lead time (min) | mCSI Pool1 EarthFormer | mCSI Pool1 DPPN-cas | mCSI Pool4 EarthFormer | mCSI Pool4 DPPN-cas | mCSI Pool16 EarthFormer | mCSI Pool16 DPPN-cas |
|---|---|---|---|---|---|---|
| 70 | 0.2981 | 0.3030 | 0.3116 | 0.3257 | 0.3401 | 0.3656 |
| 80 | 0.2763 | 0.2858 | 0.2855 | 0.3118 | 0.3085 | 0.3579 |
| 90 | 0.2570 | 0.2692 | 0.2640 | 0.2947 | 0.2833 | 0.3432 |
| 100 | 0.2396 | 0.2541 | 0.2452 | 0.2788 | 0.2619 | 0.3274 |
| 110 | 0.2240 | 0.2405 | 0.2287 | 0.2643 | 0.2445 | 0.3141 |
| 120 | 0.2089 | 0.2284 | 0.2132 | 0.2504 | 0.2275 | 0.2986 |
Skill decreases with lead time for both models. Beyond 60 minutes, most inputs to the second rollout step are forecasts from the first step, so first-hour errors carry into the second hour. Without retraining for the longer horizon, DPPN-cas scores above EarthFormer at every lead time and pooling scale, and the margin grows with lead time. At Pool16, DPPN-cas reaches 0.3432 at 90 minutes and 0.2986 at 120 minutes, against 0.2833 and 0.2275 for EarthFormer. The Pool16 gap widens from 0.0255 at 70 minutes to 0.0711 at 120 minutes, and the Pool1 gap from 0.0049 to 0.0195.
We have added these results to the revision and state throughout the manuscript that DPPN is trained for a 60-minute horizon, while the 70–120-minute results evaluate its recursive extension beyond that horizon. We agree that a sensitivity study over observation-window length and frame interval would add insight. Each setting requires retraining the corresponding models, so we list observation-window length, frame interval, and direct training for longer horizons as future work.
Response to Reviewer Cpp3
Thank you for the detailed and constructive review. In response to the reviewer's concerns, we have substantially expanded and clarified the experimental evaluation in three aspects:
Cross-dataset evaluation. We add experiments on MeteoNet to evaluate the generalization of DPPN beyond SEVIR. DPPN-Cas achieves the best CSI-M and CSI-47 performance among the compared methods under both ensemble-mean and percentile-based fusion.
Clarification of baseline metrics and evaluation availability. We reorganize the original Table~1 into two tables to distinguish published benchmark results from methods for which suitable checkpoints and implementations enable our complete evaluation. This removes the ambiguity of the ``--'' entries and makes the source and scope of each reported metric explicit.
Longer-horizon evaluation. We clarify that the primary SEVIR setting uses seven historical frames to predict six future frames at 10-min intervals, corresponding to a 60-min forecast horizon. We further extend the trained model autoregressively to 120 min without retraining and report performance from 70 to 120 min. The model retains a pixel-wise mCSI of 0.27 at 90 min and 0.23 at 120 min and a Pool16 mCSI of 0.34 at 90 min and 0.30 at 120 min. We also explicitly acknowledge that systematic sensitivity to observation-window length, frame interval, and explicit longer-horizon training remains outside the scope of the present study.
Detailed responses are provided below.