acceptodds
Under review as a conference paper at ICLR 2027

Proper Scoring Rule-based Diffusion for Probabilistic Weather Forecasting

Abstract

Recent probabilistic weather forecasters train stochastic predictors with the continuous ranked probability score (CRPS) to generate each ensemble member in a single forward pass. These models learn the predictive distribution from the forecast context alone, which becomes difficult at longer forecast horizons where uncertainty is high. To learn the predictive distribution more effectively, we introduce auxiliary conditional denoising tasks that predict the same future state from the context and its corrupted version, which provides partial future information that can reduce prediction ambiguity. Building on distributional diffusion models, we learn the conditional distributions of these tasks with a single stochastic predictor by minimizing a proper scoring rule across noise levels. At inference, the predictor can still generate each ensemble member in a single forward pass at the fully corrupted endpoint. Standard CRPS training is recovered as the endpoint-only special case of our formulation, so our framework extends existing CRPS-based forecasters with only additional conditioning inputs. Controlled experiments show that the auxiliary tasks improve one-step forecasting across architectures, with larger gains at longer forecast horizons. The gains extend to high-dimensional global weather forecasting under both training from scratch and fine-tuning, along with improved calibration and potential benefits for generalization under distribution shift.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
climate science, probabilistic weather forecasting

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_EeyKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper observes overfitting and performance degradation in existing CRPS-based probabilistic weather forecasters when directly predicting distant futures with high uncertainty. To address this, it jointly learns auxiliary conditional denoising tasks that provide a corrupted version of the future state alongside the forecast context. The intuition is that learning easier tasks with partial future information through shared parameters can benefit the original context-only forecasting task. Building on Distributional Diffusion Models (DDMs), the method learns conditional future distributions across noise levels using a CRPS-based objective, adding the corrupted future and its noise level as conditioning inputs to existing forecasters. At inference, it uses the fully corrupted endpoint (t=1t=1), which contains no future information, retaining one model evaluation per ensemble member per forecast step (NFE=1). Under matched architectures and training settings, the method reduces nCRPS for direct three-day forecasting by 9.4% for FGN and 8.1% for FCN3. In the global MOSAIC experiment with 84 forecast channels, it improves CRPS in 93.7% of the 840 channel–lead-time pairs over a ten-day rollout.

Critical strengths
  1. A coherent problem–method–evaluation narrative. The paper identifies difficulties in direct long-horizon forecasting, provides evidence of overfitting in this setting, and introduces a plausible intervention through auxiliary denoising tasks. The experiments then demonstrate that the intervention mitigates the observed degradation. This is a reasonable and well-organized narrative, although the causal connections require stronger support, as discussed below.

  2. Strong empirical performance against matched baselines. The proposed training objective delivers substantial gains over standard CRPS training in the more challenging long-horizon direct forecasting tasks. Improvements also extend to larger global forecasting models while preserving single-evaluation ensemble generation.

  3. Extensive supporting analyses. The paper provides analyses of the auxiliary-task noise range, forecast horizon, sampling steps, autoregressive rollouts, ensemble calibration, distribution shift, and tropical cyclone cases. These experiments help characterize both the benefits and the conditions under which improvements are limited.

Critical weaknesses
  1. The title and method naming do not clearly communicate the distinctive contribution. The central design choice is to use auxiliary conditional denoising tasks to improve context-only forecasting. However, neither the title nor the DDM label used in the experiments makes this contribution explicit. Since DDMs and proper-scoring-rule objectives originate in prior work, clearer terminology would help distinguish the proposed auxiliary-training strategy from the underlying framework.

  2. The proposed explanation for the gains needs more rigorous validation. The paper connects difficult long-horizon forecasting to overfitting, and motivates auxiliary tasks as a source of regularization that alleviates this problem. While the experiments support the effectiveness of the intervention, they do not firmly establish these causal links. In particular, why does overfitting become more pronounced at longer direct forecast horizons, and through what specific mechanism does auxiliary denoising regularize the shared predictor? Given the simplicity of the method, a more rigorous account of why it works would substantially strengthen the contribution. Relevant prior theory on auxiliary-task regularization, together with controlled comparisons against conventional regularization or alternative auxiliary tasks, could help substantiate this interpretation.

  3. The scope of empirical validation is narrower than the apparent applicability of the method. The proposed auxiliary-training strategy does not appear inherently specific to weather forecasting: it could potentially apply to other conditional probabilistic prediction tasks. Evaluating it in at least one additional domain would help establish whether the benefits reflect a broader learning principle or depend on properties of the weather datasets and architectures studied here. I recognize that the authors contrained the scope for this paper to weather forecast which is the one of the important forecasting task. However, the method or the motivation is not the problem only for weather case. Given that overfitting motivation and regularization method could be a problem in any forecasting problem, I believe that this paper should have some experimental results for other domains.

Questions for authors

See weaknesses section.

5 comments

We sincerely thank the reviewer for their careful review and feedback. We are pleased that the reviewer found our paper to follow a “coherent” and “well-organized” narrative. We are also grateful that the reviewer acknowledged the “strong empirical performance” against matched baselines. Furthermore, we appreciate the reviewer’s recognition of our “extensive supporting analyses”. We appreciate the reviewer’s suggestions for further strengthening the paper and address each concern in detail below.


(CW1) The title and method naming do not clearly communicate the distinctive contribution.

  • We appreciate the reviewer's suggestion to clarify our distinctive contribution relative to the prior work [1] and agree that clearer terminology would help distinguish our forecasting-oriented approach from the original DDM framework.

  • The main distinction from DDM lies in how conditional denoising distributions are used. DDM learns them for use in a reverse sampling process. Whereas our goal is to extend CRPS-based context-only weather forecasting through distributional diffusion, thereby intermediate conditional denoising tasks serve as auxiliary learning signals for the context-only forecasting at t=1t=1 without being used at one-step inference.

  • In this respect, we believe that our current title reflects this methodological formulation. However, we agree with the reviewer that referring to our approach simply as “DDM” does not clearly distinguish our forecasting-oriented contribution from the original framework. We will therefore introduce the method name D$^2$Cast (Distributional Diffusion for Weather Forecasting) and update the manuscript.

  • To avoid confusing the other reviewers during the ongoing rebuttal period, we will revise the manuscript to reflect the new method name as soon as the discussion concludes.


(CW2) In particular, (1) why does overfitting become more pronounced at longer direct forecast horizons and (2) what specific mechanism does auxiliary denoising regularize the shared predictor?

  • We first clarify the weather-specific context motivating long-horizon direct forecasting. In other applications, such as robot manipulation [1] or video generation, a common goal is to produce a coherent sequence of actions or frames at fine temporal resolution. In weather forecasting, on the other hand, there is also a strong demand for predictions at longer lead times (e.g., medium-range weather forecasts), even when densely spaced intermediate states are not needed. Direct forecasting addresses this need by predicting a distant future state without sequential rollout, avoiding repeated model evaluations and the accumulation of rollout errors.

  • Why does overfitting become more pronounced at longer direct forecast horizons? Direct forecasting requires learning a future distribution across a long interval in a single step, with uncertainty increasing due to chaotic atmospheric dynamics [2]. Yet only one realized future is available per training context. With finite data, a flexible model can lower its empirical CRPS by fitting these individual realizations without learning a conditional distribution that generalizes. We hypothesize that this tendency becomes more pronounced at longer horizons, where greater predictive uncertainty makes realization-specific variation more substantial, consistent with the stronger overfitting observed in Figure 2.

  • Through what mechanism does auxiliary denoising regularize the shared predictor? Our auxiliary tasks provide partial future information during training: the model predicts the same future X0X_{0} from the context cc and a corrupted future XtX_{t} at different noise levels. At low noise levels, realization-specific details can be recovered from XtX_{t} rather than fitted through cc. As the noise increases, XtX_{t} becomes less informative, and the model must rely more on cc. Crucially, these tasks share the same target and network parameters. We hypothesize that this shared training encourages the model to use cc for generalizable predictive structure and XtX_{t} for realization-specific details, reducing its tendency to overfit to the latter through cc at the context-only endpoint t=1t=1.

  • The training dynamics of D$^2$Cast in Figure 14 support this mechanism. At longer horizons (3d and 5d), D$^2$Cast keeps a higher training ensemble RMSE at t=1t=1 than CRPS training, yet its validation RMSE falls below even the best value of CRPS training. This indicates that the lower training error of CRPS training comes from fitting the realization-specific part, which does not generalize to unseen data. D$^2$Cast instead represents this part as ensemble spread, which remains close to its validation RMSE rather than collapsing. At shorter horizons (12h and 1d), where this part is small, the two objectives behave nearly identically. We leave a formal analysis of this mechanism to future work and will add this discussion, including Figure 14, to the revised manuscript.

  • We also conducted controlled comparisons against conventional regularization and an alternative auxiliary task; please see our response to Q1 of Reviewer BeY1 for details. Across the four tested weight-decay/dropout configurations, conventional regularization provides only marginal gains and does not close the gap to D$^2$Cast, particularly at longer horizons.


(CW3) Evaluating it in at least one additional domain would help establish …

  • To address the reviewer's suggestion to evaluate our formulation beyond weather forecasting, we additionally evaluate it on the 2D incompressible Navier–Stokes dataset, a standard benchmark for learning dynamical systems in scientific machine learning.

  • Using the Navier–Stokes dataset from [3], we compare standard CRPS training and D$^2$Cast with the same U-Net backbone and matched training settings, directly predicting xt+τx_{t+\tau} from xtx_t for τ∈{1,2,4,8}\tau \in \lbrace 1,2,4,8 \rbrace. We use 1000, 200, and 200 trajectories for training, validation, and testing, respectively. The table below reports test fair CRPS and ensemble-mean relative L2 error (10 members, NFE = 1), with Δ\Delta denoting the relative change from the CRPS baseline.

τMethodfair CRPS ↓rel. L2 (ens. mean) ↓
1CRPS0.00710.0147
Ours0.00730.0150
Δ+2.8%+2.0%
2CRPS0.02780.0470
Ours0.02720.0454
Δ-2.2%-3.4%
4CRPS0.06250.1061
Ours0.05930.1000
Δ–5.1%–5.7%
8CRPS0.12360.2026
Ours0.11480.1885
Δ−7.2%−7.0%
  • The result shows that D$^2$Cast slightly underperforms the CRPS baseline at the shortest stride but shows clear improvements at longer strides, consistent with the stronger gains at longer horizons in our weather experiments. Together with the improvements across diverse architectures (Table 14), these results suggest that the benefits of our formulation may extend beyond the specific datasets and architectures studied in weather forecasting, particularly under higher forecast uncertainty.

 [1] De Bortoli et al., Distributional Diffusion Models with Scoring Rules, ICML 2025.
 [2] Lorenz, The predictability of a flow which possesses many scales of motion, Tellus, 1969.
 [3] Li et al., Fourier Neural Operator for Parametric Partial Differential Equations, ICLR 2021.

Reviewer_9t3DOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a proper scoring rule-based diffusion framework that enhances one-step probabilistic weather forecasting. The authors transform one-step CRPS training and train it across intermediate noise levels by feeding it corrupted future states which acts as a regularizes and improves forecasts especially at longer lead times.

Critical strengths

• Innovation:

  • Reframing standard CRPS as and endpoint-only DMM acts as a strong mathematical basis.
  • Using diffusion for noise stages as auxiliary supervision and regularizes during training is a very interesting idea.

-Applicability: The presented method can be incorporated into existing probabilistic models with only minor changes and a minimal addition of parameters.

-The paper is clearly written.

-The authors performed an extensive range of experiments.

Critical weaknesses

The paper mostly adapts and combines existing methods and applies them to a practical domain. I think authors should add metrics that that assess spectral blurring like power spectral density and kinetic energy score.

Questions for authors

• Switching a baseline model to DDM requires widening the input convolutional/linear layer to take Xt and adding time-embedding modules (t), slightly increasing parameter count. How would an increase in parameters affect the performance of the models?

• In most evaluation authors use NFE = 1. How would different values of NFE > 1 affect model performance especially at longer lead times?

• Why is the noise level weighting fix uniform?

• Add rank histograms for spatial and ensemble calibration. Rank histograms reveal the exact nature of miscalibration—such as whether the ensemble is uniformly underdispersive (U-shaped), conditionally biased (asymmetric), or overdispersive (dome-shaped).

• State the used hardware and the training and inference times.

• Figure 9 is too small, when reading the printed document I was not able to read it.

4 comments

We sincerely thank the reviewer for their thoughtful evaluation and constructive feedback. We are pleased that the reviewer recognized the "strong mathematical basis" and "very interesting idea" behind our approach, as well as its practical applicability to existing probabilistic models "with only minor changes". Furthermore, we are grateful that the reviewer found our paper "clearly written" and acknowledged the "extensive range of experiments" conducted to validate our approach. We appreciate the reviewer's valuable suggestions for further strengthening the paper and address each concern below.


(CW) The paper mostly adapts and combines existing methods and applies them to a practical domain.

  • We acknowledge that our work adapts and combines existing methods, namely CRPS-based forecasters and distributional diffusion [1]. This is inherent to our goal of revisiting standard CRPS training rather than replacing it, and our novelty lies in what this revisiting reveals and enables.

  • Specifically, revisiting standard CRPS training as the endpoint-only special case of distributional diffusion allows us to extend the existing CRPS-based weather forecasters with minimal changes. The corrupted future and its noise level are added as conditioning inputs, and inference remains a single forward pass at t=1t=1. Consequently, intermediate noise levels serve as auxiliary supervision for one-step forecasting during training, whereas [1] uses them for reverse sampling.

  • Our experiments then show that this simple extension is practically useful under various settings. Since only minor changes are required and one-step inference is retained, we believe that our work is worthwhile in offering a practical extension of CRPS training that existing weather forecasters can readily adopt.


(CW) … I think authors should add metrics that that assess spectral blurring …

  • We thank the reviewer for this valuable suggestion and agree that adding spectral evaluation can help assess whether DDM keeps realistic energy across spatial scales. More details including additional figures are provided in Appendix D.4.7.

  • For MOSAIC and U-Cast, we have added kinetic energy (KE) spectra computed at 10m and 300hPa (Figure 33) and power spectral densities (PSDs) of representative variables (Figures 34-35). All spectra are computed on individual ensemble members and compared with ERA5. For 10m KE, the table below summarizes the ratio of the forecast KE spectrum to the ERA5 KE spectrum, averaged over three wavelength bands.

Table. KE ratio to ERA5 averaged within each wavelength band at 10m. Closer to 1 is better.

ModelWavelength (km)1d (CRPS / DDM)3d (CRPS / DDM)10d (CRPS / DDM)
MOSAIC1333–10001.004 / 1.0000.996 / 0.9950.984 / 0.988
1000–5000.999 / 0.9970.991 / 0.9910.985 / 0.986
500–3330.992 / 0.9970.982 / 0.9920.977 / 0.987
U-Cast1333–10001.000 / 1.0020.993 / 0.9980.987 / 0.992
1000–5000.991 / 0.9960.987 / 0.9930.984 / 0.991
500–3330.987 / 0.9930.985 / 0.9910.982 / 0.990
  • As illustrated in Figures 33-35, neither objective shows notable spectral blurring and spectral characteristics tend to depend more on the forecasting backbone than on the training objective. Within each backbone, DDM generally achieves slightly better spectral agreement with ERA5. These results indicate that the improvements of DDM in forecast skill are therefore not achieved through additional spectral smoothing, but rather while preserving or modestly improving the spectral fidelity of the CRPS-trained models.

(Q1) Switching a baseline model to DDM requires … slightly increasing parameter count. How would an increase in parameters affect the performance of the models?

  • Thanks for raising this question which helps us to clarify the source of our performance gains. To isolate the benefits of auxiliary denoising training, we keep changes to existing CRPS-based forecasters minimal so that gains over standard CRPS training can be attributed to the training scheme rather than to architectural changes or added capacity.

  • Specifically, we retain the original forecasting backbones, only widening the input projection for the corrupted future state XtX_t and introducing the tt-conditioning module for noise-level tt. Hence, the parameter increase is negligible, only about 0.02–0.09% in the pilot experiments, and about 1~3% for main experiments. To investigate this minimal parameter increase, we further trained CRPS models with about 1% more parameters, and the change in performance was negligible (Table 19).

  • Beyond this minimal setting, further gains may be possible with alternative designs or more parameters for processing XtX_t and tt. iDDM [2], a concurrent follow-up to DDMs, shows that such design choices can affect performance in image generation. As our focus is on the benefits of auxiliary denoising training with minimal architectural changes, we leave this exploration to future work.


(Q2) In most evaluation authors use NFE = 1. How would different values of NFE > 1 affect model performance especially at longer lead times?

  • Our primary goal is to achieve improved forecasting performance over standard CRPS training at nearly identical inference cost. We therefore focus on NFE = 1 in our main evaluations.

  • In the idealized setting where the joint conditional denoising distributions are recovered exactly at all noise levels, DDM sampling recovers the target predictive distribution regardless of NFE (Appendix B.1).

  • In practice, extra NFEs may be beneficial when accurate one-step prediction is challenging, as in the 3d direct forecasting task of our pilot study (Figure 5). In the high-dimensional autoregressive rollout, NFE = 2 also yields modest gains for MOSAIC at every lead time, which become smaller as accumulated rollout errors limit the benefit of multi-step sampling (Figure 20).

  • These modest gains are consistent with our choice of λ=1\lambda=1 (which preserves the connection with standard CRPS training), as the original DDM study reports the best results when λ=1\lambda=1 for few-step sampling and better performance with λ<1\lambda<1 when the number of steps increases [1]. Exploring such settings to investigate the potential advantages of multi-step sampling is an interesting direction for future work.


(Q3) Why is the noise level weighting fix uniform?

  • We choose uniform weighting as the simplest extension of standard CRPS training to all noise levels. We agree that non-uniform weighting may offer further improvements, as noise-level weighting and sampling have been shown to affect generation quality in diffusion models [3]. More directly, concurrent work iDDM [2] reports additional improvements in the DDM framework by adjusting the noise-level sampling distribution, which effectively reweights the noise levels.

  • Our goal, however, is to clearly assess the benefits of generalizing standard CRPS training within the DDM framework (i.e. introducing and learning auxiliary denoising tasks), rather than optimizing the noise-level weighting strategy. In this work, we therefore deliberately adopt uniform weighting to minimize additional design choices and provide a simple, controlled extension of standard CRPS training.


(Q4) Add rank histograms for spatial and ensemble calibration.

  • We thank the reviewer for this suggestion, and we agree that rank histograms further strengthen our calibration analysis. More details including additional figures are provided in Appendix D.4.8.

  • Following GenCast [4], we extended the calibration analysis in the main paper with rank histograms at 1-, 3-, and 10-day lead time (Figure 36 for MOSAIC and Figure 37 for U-Cast). Consistent with the spread–skill results there, DDM generally yields slightly flatter histograms than CRPS training for both backbones.

  • To assess spatial calibration, we additionally report spatially pooled rank histograms, where forecasts and targets (ERA5) are average-pooled at pool sizes of 481, 962, 1922, and 3828km, inspired by the pooling scheme in GenCast [4], before ranks are computed. The same trend holds across pool sizes, suggesting that the calibration gains extend beyond individual grid points to spatial aggregates (Figures 38-39).


(Q5) State the used hardware and the training and inference times …

  • Thank you for this helpful suggestion, and we agree with the reviewer that reporting computational costs is important for assessing the practicality of our method.

  • We report the hardware and wall-clock training and inference times below, under matched training configurations for CRPS and DDM. Inference times are measured for a single ensemble member over a 10-day rollout (10 autoregressive steps for MOSAIC and 20 for U-Cast), using NFE=1 for DDM. As shown in the table below, DDM introduces only minor computational overhead in both training and inference. The table is now included in Appendix C.5.

SettingHardwareCRPSDDMOverhead
Training time (h)
MOSAIC8 × B20033.133.8+2.1%
U-Cast pretraining†4 × B20026.426.4–
U-Cast fine-tuning4 × B2006.06.2+3.3%
Inference time (s)
MOSAIC1 × B2001.9592.022+3.2%
U-Cast1 × B2007.2577.335+1.1%

† Pretraining with MAE (Mean Absolute Error) is shared by both objectives.


(Q6) Figure 9 is too small …

  • We thank the reviewer for pointing out this. Figure 9 was reduced in size due to the page limit. With the additional page allowed in the revision, we have splitted Figure 9 into two figures (now, Figures 9-10) to improve readability in the revised manuscript.

 [1] De Bortoli et al., Distributional Diffusion Models with Scoring Rules, ICML 2025.
 [2] Martorella et al., Improved Distributional Diffusion Models, 2026.
 [3] Karras et al., Elucidating the Design Space of Diffusion-Based Generative Models, NeurIPS 2022.
 [4] Price et al., Probabilistic weather forecasting with machine learning, Nature, 2025.

Dear Reviewer,

As the discussion period draws to a close, we wanted to briefly follow up on our responses to your comments. We hope they have addressed your concerns and would appreciate any further feedback you may have.

Thank you again for your time and consideration.

Reviewer_BeY1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper adapts DDM to weather forecasting. A stochastic predictor receives weather context, a corrupted future state, and its noise level, and is trained with coordinate-wise fair CRPS. Inference uses only the fully corrupted endpoint, requiring one evaluation per member per forecast advance without future observations. Intermediate denoising tasks provide auxiliary supervision. Validation covers four lower-resolution backbones and two higher-dimensional models, including training from scratch and fine-tuning. The strongest gains occur at longer direct horizons. Additional evidence includes endpoint/noise-range ablations, spread–skill diagnostics, ERA5-to-HRES transfer, and cyclone cases.

Critical strengths
  1. The empirical question is controlled and practically relevant. The method improves training while retaining one model evaluation per ensemble member and forecast advance. Results span FGN, FCN3, Swin, U-Net, MOSAIC, and U-Cast. Table 1 reports three-day direct-forecast nCRPS reductions of 8.1% for FCN3 and 9.4% for FGN. These are meaningful improvements, and the architectural breadth makes the contribution stronger than a single-model application.

  2. The endpoint ablation goes beyond controlling additional parameters. Appendix D.2 compares endpoint-only and intermediate-noise training while retaining the same added inputs and time conditioning. The respective improvements are approximately 0.47% versus 8.10% for FCN3, and 0.41% versus 9.96% for FGN. Checkpoints are selected using validation nCRPS. Together, these controls reduce concerns that the gains arise merely from added modules or comparison against a degraded final baseline checkpoint, although they do not rule out alternative regularization recipes.

Critical weaknesses
  1. The incremental methodological contribution would benefit from stronger alternative-training comparisons. The objective and stochastic distributional denoising mechanism substantially inherit DDM, as the authors acknowledge. The main additional contribution therefore rests on the empirical value of intermediate denoising tasks for weather forecasting. The matched training recipes and the improvements across all tested DDM corruption configurations at longer forecast intervals are reassuring. What remains unclear is whether the gains persist against a comparably tuned CRPS baseline with stronger conventional regularization, or a simpler auxiliary reconstruction objective. Appendix D.3.2 explores multiple DDM noise configurations, but comparable exploration of alternative CRPS training recipes is not established. This is an unresolved attribution question, rather than evidence that the reported comparisons are unfair.

  2. The evidence does not yet establish the reliability and breadth of the probabilistic forecasting improvements. Independent training repetitions and confidence intervals are not reported. Some short-horizon and U-Cast improvements are extremely small or tied at displayed precision; correlated channel–lead-time win counts cannot replace uncertainty estimates. Moreover, coordinate-wise CRPS and spread–skill diagnostics do not fully characterize spatial and cross-variable dependence. The sampling argument in Appendix B.1 assumes recovery of the joint conditional distribution, which the marginal objective alone does not guarantee. Tables 17–18 provide useful quantitative cyclone evidence, including track CRPS and ensemble-member proportions within specified radii, but cover only two storms. These results therefore do not yet establish broader improvements in joint weather-field quality or calibration. This matters because reliable empirical improvement, rather than a fundamentally new algorithm, is the principal basis for acceptance.

Questions for authors
  1. Clarification and additional evidence needed for the current contribution claims: The matched training recipes, endpoint control, and robustness across tested DDM noise configurations are useful evidence. What were the respective hyperparameter-search budgets for CRPS and DDM? On representative short- and long-horizon settings, does the advantage persist against a reasonably tuned CRPS regularization recipe and one simple auxiliary reconstruction alternative under comparable training budgets? The existing endpoint control should be retained in this comparison.

  2. Additional evidence needed for the current reliability claims: Can you provide independent training repeats for representative settings and paired skill-difference intervals that account for temporal dependence between forecast initializations? Please also report unrounded effect sizes for U-Cast. This would help determine whether the small improvements supporting transfer to pretrained models are reproducible.

  3. Additional evidence needed for the current joint-field forecasting claims: Tables 17–18 already provide quantitative evidence for two cyclones. Using existing forecast samples, could you assess dependence-sensitive quantities across a broader set of forecast cases? For example, spatial aggregation should be performed separately within each ensemble member, followed by CRPS evaluation of the resulting ensemble of regional means or sums against the corresponding observed aggregate. Simply averaging pointwise CRPS across grid cells would not test spatial dependence. Multivariable derived quantities or joint events could provide complementary evidence, together with reliability or coverage diagnostics. This would help establish how far the improvements extend beyond marginal forecast quality and the two cyclone cases.

7 comments

We thank the reviewer for their thoughtful assessment and constructive feedback. We are particularly pleased that the reviewer recognized our empirical investigation as "controlled and practically relevant" and acknowledged the "meaningful improvements". We also appreciate the reviewer's recognition of the "architectural breadth" of our experiments and the value of our endpoint ablation. Furthermore, we are grateful that the reviewer rated our presentation as "clear and easy to follow". We address the remaining concerns through clarifications and supporting evidence below.


(Q1) … does the advantage persist against a reasonably tuned CRPS regularization recipe and one simple auxiliary reconstruction alternative under comparable training budgets?

  • For the 1.5° experiments, the CRPS baselines use their own official configurations without modification, recommended by their original authors, which already include regularization (weight decay in MOSAIC and U-Cast, and dropout in U-Cast). Due to the high computational costs (see our response to Q5 of Reviewer 9t3D), we apply DDM on top of the identical configurations without re-tuning the shared optimization and regularization hyperparameters. Thus, the 1.5° experiments evaluate the benefit of adding our auxiliary objective to the existing regularized training configurations, rather than comparing separately tuned training recipes.

  • Conventional CRPS regularization: Developing a method naturally involves exploring design choices; in our case, we explored four corruption configurations (Table 17). As the reviewer noted, this can be viewed as additional tuning. To test whether a comparable tuning budget spent on conventional regularization could yield similar gains, we additionally evaluated four CRPS configurations: weight decay ∈{1e−4,1e−2}\in \lbrace 1e-4, 1e-2 \rbrace × dropout ratio ∈{0,0.1}\in \lbrace 0, 0.1 \rbrace for both FGN and FCN3. The table below shows that conventional regularization yields marginal gains and no configuration closes the gap to DDM, especially at longer horizons.

Table: Evaluation test nCRPS (lower is better) of CRPS training with conventional regularization (weight decay, dropout). The WD = 1e-4, dropout = 0 rows represent the paper settings. Δ (%) denotes the relative change of DDM with respect to the best CRPS result in each column.

ModelObjectiveWDDropout12h1d3d5d
FGNCRPS1e-400.058130.079210.186830.24627
CRPS1e-200.058420.078380.186140.24550
CRPS1e-40.10.059230.081620.192680.24707
CRPS1e-20.10.059030.081260.192570.24644
DDM1e-400.057660.077750.169230.22898
Δ vs. best CRPS (%)−0.8−0.8−9.1−6.7
FCN3CRPS1e-400.073580.096700.191460.24521
CRPS1e-200.073550.096540.191080.24487
CRPS1e-40.10.074910.098890.192270.24516
CRPS1e-20.10.074910.098640.192110.24514
DDM1e-400.073500.094230.175950.22833
Δ vs. best CRPS (%)−0.1−2.4−7.9−6.8
  • Auxiliary reconstruction alternative: To further examine whether the gains arise from simple auxiliary reconstruction, we evaluate a reconstruction alternative inspired by the Masked AutoEncoder [1] at the 3d horizon. Specifically, we mask a random fraction ρ∈[0,1]\rho \in [0,1] of the future state to zero and train the model to predict the full future state. ρ\rho is fed into the tt-conditioning module, so the architecture and the training objective remain identical to those of DDM and only the corruption mechanism changes. Under matched training settings, we compare this with the CRPS baselines, the endpoint-only control and DDM.

Table: Evaluation test nCRPS (lower is better). Numbers in parentheses are the relative change (%) from the paper CRPS setting.

ConfigurationFGN nCRPS ↓FGN nRMSE ↓FCN3 nCRPS ↓FCN3 nRMSE ↓
CRPS (paper)0.186830.384650.191460.39205
CRPS (best reg.)0.18614 (−0.37)0.38281 (−0.48)0.19108 (−0.20)0.39127 (−0.20)
Masked recon.0.17572 (−5.95)0.36575 (−4.91)0.18123 (−5.34)0.37619 (−4.05)
endpoint-only0.18607 (−0.41)0.38278 (−0.49)0.19056 (−0.47)0.39062 (−0.36)
DDM0.16923 (−9.42)0.35423 (−7.91)0.17595 (−8.10)0.36637 (−6.55)
  • The masked reconstruction alternative clearly improves over CRPS training and outperforms conventional regularization, suggesting that learning from partially corrupted future states provides useful auxiliary learning signals. Nevertheless, it falls short of DDM, indicating that our formulation offers additional benefits beyond this simple reconstruction-based alternative.

(Q2-1) Can you provide independent training repeats for representative settings … Please also report unrounded effect sizes for U-Cast …

  • To assess reproducibility, we repeated training with three random seeds for both the CRPS and DDM objectives on FGN and FCN3 in the pilot setting and U-Cast at 1.5°. As requested, we report the U-Cast results at higher precision, with relative changes Δ (%) computed from full-precision scores. Per-seed results are provided in Appendix D.3.3 and D.4.3, respectively.

  • Pilot setting: The two objectives perform comparably at 12h, while DDM improves both nCRPS and nRMSE in every seed at 3d, consistent with the original results (Table 1).

Table: Evaluation test nCRPS and nRMSE (lower is better), reported as mean ± std across three seeds. Δ (%) denotes paired seed-wise relative changes (negative means DDM is better).

ModelObjectivenCRPS 12h ↓nCRPS 3d ↓nRMSE 12h ↓nRMSE 3d ↓
FGNCRPS0.05813 ± 0.000260.18620 ± 0.000800.11385 ± 0.000420.38306 ± 0.00163
FGNDDM0.05789 ± 0.000240.16901 ± 0.000820.11346 ± 0.000490.35400 ± 0.00184
Δ (%)−0.42 ± 0.41−9.23 ± 0.22−0.35 ± 0.35−7.59 ± 0.39
FCN3CRPS0.07359 ± 0.000010.19148 ± 0.000290.14624 ± 0.000080.39223 ± 0.00038
FCN3DDM0.07356 ± 0.000070.17599 ± 0.000360.14621 ± 0.000160.36657 ± 0.00070
Δ (%)−0.03 ± 0.08−8.09 ± 0.33−0.02 ± 0.11−6.54 ± 0.26
  • U-Cast at 1.5°: We focused our additional experiments on U-Cast, which allows efficient fine-tuning from a pretrained checkpoint. Given the modest gains of DDM under the 8-epoch original schedule [2], we extended fine-tuning to 12 epochs to assess reproducibility and the effect of longer training together.

Table: Evaluation CRPS (lower is better) under CRPS and DDM fine-tuning objectives. The 8-epoch results are from the original single run (Table 3) and the 12-epoch results report mean ± std across three seeds.

EpochsObjective1d Z5001d T8501d Q7003d Z5003d T8503d Q70010d Z50010d T85010d Q700
8CRPS20.44780.28100.229056.09300.47960.3731260.10851.32610.7037
DDM20.42910.28040.228856.05370.47550.3716259.46071.31430.6943
Δ (%)−0.09−0.21−0.08−0.07−0.84−0.39−0.25−0.89−1.35
12CRPS20.1551 ± 0.10290.2764 ± 0.00010.2275 ± 0.000155.3624 ± 0.13090.4723 ± 0.00090.3693 ± 0.0002258.4723 ± 0.28511.3206 ± 0.00070.7027 ± 0.0011
DDM19.9918 ± 0.02220.2759 ± 0.00040.2271 ± 0.000255.0131 ± 0.21960.4688 ± 0.00060.3679 ± 0.0001257.1318 ± 0.60711.3085 ± 0.00160.6938 ± 0.0011
Δ (%)−0.81 ± 0.51−0.21 ± 0.11−0.18 ± 0.04−0.63 ± 0.61−0.73 ± 0.30−0.38 ± 0.05−0.52 ± 0.34−0.92 ± 0.17−1.26 ± 0.26
  • The three 12-epoch U-Cast runs achieve lower CRPS than the original 8-epoch run on average under both objectives. DDM consistently outperforms CRPS across all nine targets in every seed (Table 20), suggesting that even the modest gains from transferring DDM to a pretrained model are reproducible across training seeds.

  • We hypothesize that these modest gains may reflect U-Cast's 12-hour forecast horizon (consistent with smaller short-horizon gains in the pilot setting) and extensive pretraining (100 epochs), which may reduce the incremental benefit of auxiliary learning signals over CRPS fine-tuning.

(Q2-2) Can you provide ... paired skill-difference intervals that account for temporal dependence between forecast initializations?

  • To account for temporal dependence between forecast initializations, we computed paired skill-difference intervals for each channel and lead time following the significance-testing protocol of GenCast [3]. Further details are provided in Appendix D.4.5 and the table below summarizes the results, classifying intervals as favoring DDM, not significant, or favoring CRPS.
ModelLead timesTotal CellsDDM BetterNot significantCRPS Better
MOSAIC1–10 days8407624632
U-Cast0.5–10 days16801210254216
  • DDM significantly improves 91% (762/840) of the cells for MOSAIC and 72% (1210/1680) for U-Cast. Additionally, Figures 28–29 show significant gains across all lead times for the representative variables in MOSAIC and for most variables beyond the first day in U-Cast, suggesting the statistical significance of DDM improvements.

  • Together with Q2-1, these results support that the DDM gains are reproducible across seeds and statistically significant under temporally dependent evaluation.

(Q3) Using existing forecast samples, could you assess dependence-sensitive quantities across a broader set of forecast cases?

  • We agree that coordinate-wise CRPS does not fully characterize spatial and cross-variable dependence. To assess dependence, we evaluated two dependence-sensitive quantities following the protocol of GenCast [3] and FGN [4]. More details including additional results are provided in Appendix D.4.6.

Spatial dependence: We evaluate average- and max-pooled CRPS over spatial windows of increasing size. Across T850 (shown below) and other variables (Figures 30-31), DDM generally achieves lower pooled CRPS for both MOSAIC and U-Cast, indicating that the improvements persist under spatial aggregation.

Table: Spatially average- and max-pooled CRPS for T850 (lower is better).

ModelPool sizeObjectiveAvg 1dAvg 3dAvg 10dMax 1dMax 3dMax 10d
MOSAIC481 kmCRPS0.2210.3901.2160.2780.4431.242
DDM0.2140.3791.2020.2680.4311.226
Δ (%)−3.36−2.88−1.15−3.50−2.83−1.23
1,922 kmCRPS0.1060.2040.8520.2580.3810.982
DDM0.1030.1990.8420.2490.3710.969
Δ (%)−3.15−2.56−1.25−3.50−2.63−1.30
3,828 kmCRPS0.0670.1210.5110.2480.3470.792
DDM0.0650.1200.5060.2400.3390.781
Δ (%)−3.92−1.04−0.97−3.34−2.43−1.41
U-Cast481 kmCRPS0.2140.3961.2410.2640.4441.261
DDM0.2120.3911.2300.2630.4401.251
Δ (%)−0.53−1.05−0.86−0.26−0.83−0.83
1,922 kmCRPS0.1080.2140.8720.2470.3810.996
DDM0.1070.2110.8640.2460.3780.987
Δ (%)−1.03−1.49−0.93−0.26−0.58−0.86
3,828 kmCRPS0.0690.1300.5220.2380.3480.808
DDM0.0680.1280.5180.2370.3470.800
Δ (%)−1.24−1.53−0.82−0.27−0.47−0.96

Cross-variable dependence: We evaluate cross-variable CRPS of derived quantities that depend on the joint distribution across variables or pressure-levels, namely 10m wind speed (WS10) and the geopotential thickness Z300 − Z500. DDM achieves lower CRPS for both quantities across all evaluated lead times for MOSAIC and U-Cast (Figure 32).

Table: CRPS for derived quantities WS10 and Z300 − Z500 (lower is better).

ModelQuantityObjective1d3d10d
MOSAICWS10CRPS0.34020.55331.1326
DDM0.32610.53701.1246
Δ (%)−4.13−2.94−0.71
Z300 − Z500CRPS22.3445.73148.60
DDM21.7044.18146.01
Δ (%)−2.89−3.41−1.74
U-CastWS10CRPS0.32610.55241.1593
DDM0.32550.54821.1503
Δ (%)−0.17−0.75−0.78
Z300 − Z500CRPS21.1144.94150.48
DDM21.0244.45148.92
Δ (%)−0.43−1.09−1.04
  • Together, these results suggest that the improvements from DDM carry over to spatially aggregated and cross-variable derived quantities that depend on the joint distribution.

(CW2) The sampling argument in Appendix B.1 assumes recovery of the joint conditional distribution, which the marginal objective alone does not guarantee.

  • We would like to clarify that Appendix B.1 does not claim that component-wise CRPS guarantees recovery of the joint conditional distribution, and we do not use it as evidence of joint field quality. Rather, it analyzes the behavior of the sampler under the idealized assumption of exact joint recovery, as an intuition for interpreting the NFE results in Section 4.1. We will state this scope more clearly in Appendix B.1 and Section 3.2, and we report the evaluation of joint fields through spatial and cross-variable dependence in our response to Q3.

 [1] He et al., Masked Autoencoders Are Scalable Vision Learners, CVPR 2022.
 [2] Cachay et al., U-Cast: A Surprisingly Simple and Efficient Frontier Probabilistic AI Weather Forecaster, ICML 2026.
 [3] Price et al., Probabilistic weather forecasting with machine learning, Nature, 2025.
 [4] Alet et al., Skillful joint probabilistic weather forecasting from marginals, arxiv, 2025.

Comments on the paper

1 comment

We thank all reviewers for their positive evaluations and are glad that our additional results and clarifications during the discussion period resolved their concerns. We also believe our responses have addressed the questions from Reviewer 9t3D, who gave a positive initial assessment, and hope that these clarifications further strengthen their favorable view of our work. We have incorporated all possible revisions into the updated manuscript to date and hope these updates will be helpful for reaching the final decision. Thank you to all reviewers and Chairs for your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.