- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes JWS, a single-stage, end-to-end diffusion model for probabilistic precipitation nowcasting that operates directly in radar space with a hierarchical transformer backbone. The authors introduce two components. Masked Asynchronous Diffusion (MAD) is a timestep-sampling scheme that keeps context frames clean while assigning correlated, resolution-aware noise levels to the predicted frames. Simple Distributional Diffusion Models (sDDM) train the denoiser with a timestep-dependent interpolation between CRPS and squared error to enable few-step sampling. On SEVIR and MeteoNet, the authors report state-of-the-art CRPS and SSIM, improved calibration, and lower inference latency and fewer parameters than multi-stage latent baselines such as CasCast and FREUD.
Critical strengths
- Simple, well-motivated design. JWS is a single end-to-end pixel-space diffusion model built on a standard hierarchical transformer. It avoids separately trained compression or deterministic-prior stages, and with them the reconstruction uncertainty those stages introduce.
- Effective few-step generation. The simple distributional objective (Eq. 7) removes the tuned schedules of DDM/iDDM. Controlled comparisons on the same architecture (Fig. 6; Fig. 7, right) show clear gains over Flow Matching, Shortcut models and FGN at low NFE, and one-step JWS-B produces a 10-member ensemble in under a second.
- Strong probabilistic skill. JWS achieves the best CRPS and SSIM on both SEVIR and MeteoNet (Tables 4–5). The T/S/B variants are better calibrated than CasCast and FREUD, with lower RI and flatter rank histograms (Fig. 12).
- Parameter efficiency. The 9M-parameter JWS-T already attains lower CRPS than FREUD (521M) and CasCast (402M).
- Flexible conditioning. MAD supports variable context, zero-shot deterministic priors and zero-shot extrapolation to 36 frames (Figs. 10, 14, 19).
Critical weaknesses
-
Unclear method presentation. Fig. 2 only contrasts the FREUD and JWS pipelines, repeating Fig. 1, and does not show how JWS works: context handling, MAD noise levels, particle expansion, ξ injection, the loss and inference. Key components are delegated to an anonymous concurrent submission (iDDM), and sDDM differs from iDDM only in the loss interpolation (Eq. 7). The paper does not specify in which space and with which estimator the CRPS term is computed. It also does not explain the role of ξ at inference, where particles are dropped.
-
Accuracy and speed come from different configurations. Table 4 uses 50 NFE, while latency is compared at 20 NFE. The statement that JWS remains faster at 50 NFE seems to hold only for the smaller variants. JWS-L, which provides the best CRPS/SSIM, takes 2.41s at 20 NFE, or ≈6 s at 50 NFE if latency scales linearly, versus 3.37 s for CasCast. The 17× speed-up refers to JWS-T, and the best RI also comes from the small variants: JWS-L's RI is worse than FREUD's.
-
Gains are limited to metrics aligned with the loss. Without CFG, all JWS variants trail CasCast and FlowCast in HSS and CSI (e.g., CSI 0.4183 for JWS-L vs. 0.4401 and 0.4651). CFG adds an unconditional pass per step, and its latency is not reported. The improvements are mainly in CRPS, which is also the training objective.
-
Baselines are not evaluated under a common protocol. Most baseline numbers are quoted from three papers, although all methods are said to be compared with 10 members. Because the CRPS estimator in App. C.1 is not the fair one, CRPS depends on ensemble size. RI is recomputed while the other metrics are quoted. PreDiff is trained at 128², and different FREUD variants appear in Table 4, Fig. 9 and Fig. 13. In addition, JWS trains on the merged training and validation sets with exhaustive subsequence sampling.
-
Compute is under-reported. JWS-L requires 156.8 inference TFLOPs, 2.6× CasCast (Fig. 13), and the conditions under which the baselines were timed are not described. The training GPU type and GPU-hours are not reported, and the FLOP estimate behind the 44× training-efficiency claim is not explained. JWS-L also needs an additional pretraining phase on 128² crops .
-
Possible test-set model selection. The validation set is merged into training. The 512 samples used for ablations and for choosing the training length therefore appear to come from the test set. Moreover, temporal consistency is not evaluated reported.
Questions for authors
- Could you add a figure showing the full JWS pipeline: context frames, MAD noise levels, particle expansion, ξ injection, the loss and inference?
- How does sDDM differ from iDDM (Anonymous, 2026) beyond Eq. 7, and in which space and with which estimator is the CRPS term computed?
- What are the latency and FLOPs of each variant at the 50 NFE used in Table 4, including the CFG setting?
- Were all baselines evaluated with the same test set, ensemble size, NFE and training data? If not, could you re-evaluate them with your pipeline?
- What GPU type and how many GPU-hours were used for training, and how were the training FLOPs in Fig. 1 computed?
- Is the 512-sample subset used for ablations and for choosing the training length drawn from the test set?
- Could you report a temporal-consistency metric and explain the abrupt change at +50 min in Fig. 32? If the authors can satisfactorily address the questions above, I am willing to raise my score.
5 comments
Response 2/2
W4 & Q4 - Evaluation under a common protocol
Our evaluation closely follows CasCast. We use the same test split and metrics computed with CasCast’s own SEVIRSkillScore class (https://github.com/OpenEarthLab/CasCast/blob/master/utils/metrics.py). This aligns our task and metric implementation with prior work. In Tab. 4, we quote reported values to include baselines without public checkpoints and present each method with previously established performance.
The new Tab. 13 (appendix) additionally compares CasCast, FlowCast, FREUD, and JWS using our pipeline on the same held-out test windows, with a fixed ensemble size and NFE across models. Reproduced baseline numbers deviate from the reported ones in both directions, so there is no bias in favor of our method. JWS models achieve the lowest CRPS, the highest SSIM, the lowest RI, and with CFG the highest HSS and CSI. We added FREUD-L to the top Fig. 9 and Fig. 14.
Regarding training data, all JWS models are trained on the same data as FREUD (Schusterbauer et al., 2026).
W5 - Timing and FLOP calculation
We added details to the supplement (Sec. C.2). Tab. 7 reports GPU type (GH200), GPU counts, and training times. Table 17 breaks down training FLOPs per model, measured as described in Sec C.2. The 44x claim compares JWS-T with FREUD, which JWS-T outperforms in CRPS. Appendix C.2 now describes the timing and FLOP measurement setup. Crop pretraining is included in all reported compute, but direct full-resolution training is competitive with comparable compute (Fig. 18).
W6 & Q6 - Ablation dataset
All model-selection decisions were made on a single fixed subset of 512 randomly selected SEVIR test windows (4.2%). Full-test evaluation was performed only after these choices were finalized and did not inform model selection.
To assess performance on events excluded from model selection, we removed every SEVIR test event containing any of these 512 windows, retaining 10,674 windows (87.8%), and evaluated JWS, CasCast, FlowCast and FREUD on this held-out set with the same pipeline (Table 13, appendix). JWS performance remains stable relative to the full test set. JWS variants achieves the best score on every metric.
Moreover, MeteoNet (Table 5) provides an additional check that is independent of model selection. We applied the same recipe selected on SEVIR unchanged.
W6 & Q7 - Temporal consistency
Thank you for spotting the mistake in Fig. 32 (now Fig. 39). When assembling the figure, the +50 min panel was taken from a different forecast.
Although the sDDM loss is applied per pixel, spatio-temporal consistency is learned by the backbone, as with a pixel-wise Flow Matching objective. We now evaluate spatio-temporal consistency explicitly in Table 12. JWS variants achieve the best scores on five of six metrics. On the change ratio, JWS is slightly too static, yet achieves better results than FlowCast and FREUD-L.
Dear Reviewer LKzc,
Thank you again for your time and detailed feedback. Do our additional experiments and clarifications address your concerns? We are more than happy to clarify any remaining questions.
Response 1/2
Dear Reviewer LKzc,
We thank you for highlighting our “simple, well-motivated design,” enabling few-step generation, as well as our “strong probabilistic skill” with JWS. Below, we answer your remaining questions.
W1 & Q1 & Q2 - Pipeline Details
Thank you for the suggestion. We added Fig. 13 (appendix), which shows the full JWS pipeline used during training and inference. At inference, no particles are expanded: each ensemble member is generated with its own initial noise and xi. Thus, xi is held fixed for all ODE steps.
We added details on the sDDM loss in section A, appendix. To summarize, we use the default biased ensemble estimator of CRPS during training. CRPS and MSE are calculated per pixel in velocity space. iDDM requires the design and tuning of two timestep-dependent schedules. sDDM replaces them with a fixedinterpolation between CRPS and MSE. Although simpler, it consistently performs better across all metrics (Fig. 6).
W2 & Q3 & W3 - Latency and FLOPs
Tab. 15 reports wall-clock time and inference FLOPs. We also added Fig. 28, showing JWS latency over NFE including CFG. JWS-L takes 6.03s without guidance (12.09 s with guidance, Fig. 28), which is comparable to CasCast at matched settings (6.15 s without and 11.64 s with CFG, fp32) and 2-8x faster than FREUD and FlowCast. The 17x speed-up refers to JWS-T relative to the default CasCast configuration.
W2 - JWS-L RI
We looked further into why the largest model is less calibrated than FREUD. Fig. 18b shows that calibration depends on the training stage. After crop pretraining, JWS-L reaches an RI of 0.092 at 150k steps, below FREUD (0.136). With further full-resolution training, RI rises. The deterioration with additional training is consistent with overfitting and increasing ensemble underdispersion (Fig. 19c).
W3 - Gains beyond loss alignment
The improvements of JWS are not limited to CRPS. JWS also leads on SSIM and RI (T/S/B). With guidance, it achieves the best HSS on SEVIR and, at high thresholds, the best CSI (Fig. 26).
Moreover, the gains cannot be explained solely by alignment between objective and evaluation metric.