- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Xaurora converts the deterministic Aurora-Small model into a generative weather forecaster using denoising stochastic interpolants. The network defines an SDE drift conditioned on historical states, intermediate sampling states, and pseudo-time, trained using a replay buffer on fair-CRPS loss (requiring 42 hours on 16 A100s). Generating one ensemble member's 15-day forecast at 6-hour intervals takes approximately 13 minutes on a single A100 GPU. On 56 test initializations from 2022 at 0.25° resolution, Xaurora yields lower ensemble-mean errors than both Aurora-Small and Aurora-Large across most evaluated variables and lead times. The spectral analysis also identifies a persistent noise floor and insufficient ensemble spread at large spatial scales.
Critical strengths
-
A coherent and empirically useful extension of a pretrained forecaster. The endpoint-prediction formulation provides a natural way to initialize from Aurora, while the extension of replay-buffer training keeps the training cost under control. The reported improvements over the deterministic prior and the optimization ablations support the practical value of the complete approach. The evaluation provides evidence that the proposed adaptation can support useful ensemble forecasts.
-
Informative analysis of the method's limitations. The spectral diagnostics show that large-scale underdispersion can coexist with excess small-scale variability, which aggregate spread-skill ratios can conceal. The noise-scaling and spatial-correlation interventions further investigate this behavior and report trade-offs rather than claiming that the calibration problem is solved. This analysis adds scientific value beyond reporting aggregate forecast scores.
Critical weaknesses
-
The unit gaussian noise use in paper does not match up with weather forcasting task. Per-variable normalization addresses numerical scales but does not establish that the same normalized noise amplitude is appropriate relative to each variable's short-range conditional uncertainty. Overall variability can differ substantially from uncertainty about the next six hours. The independent grid-based noise also does not directly encode spatial or cross-variable relationships. The reported noise floor and scale-dependent miscalibration show that the resulting uncertainty is not uniformly satisfactory. Appendix B.1 also shows that noise interventions have variable-dependent effects and do not resolve the problem across lead times.
-
The parameterization conclusion exceeds what Figure 2 establishes. Figure 2 inserts the same endpoint predictor directly into endpoint, drift, and noise output roles. Its result supports the convenience of direct endpoint initialization, but does not exclude drift or noise predictors initialized using the corresponding algebraic transformations. The claim that only the proposed parameterization can preserve the prior should therefore be restricted to the tested initialization. This distinction can be addressed by more precise claims without negating the empirical contribution.
Questions for authors
-
Calibration interpretation: Based on the existing spectral diagnostics and noise interventions, which variables, spatial scales, and lead times retain the most consequential miscalibration? Please clarify the scope of the calibration claims and how the reported aggregate scores should be interpreted under these limitations.
-
Parameterization claim: Is Figure 2 intended only to compare direct reuse of an endpoint predictor without output transformations? Please distinguish this claim from an advantage over alternatives initialized to preserve the same dynamics through the transformations in Equation (6) and Appendix A.5.
2 comments
Official response [2/2]
The parameterization conclusion exceeds what Figure 2 establishes. Figure 2 inserts the same endpoint predictor directly into endpoint, drift, and noise output roles. Its result supports the convenience of direct endpoint initialization, but does not exclude drift or noise predictors initialized using the corresponding algebraic transformations.
Correct, the model output could be transformed into a drift term (Eq. 6) and supervised with drift matching (Eq. 3). Appendix A.5.1 establishes that output transformation is not necessary to train in a different output space. It can be achieved implicitly by weighting the denoising loss function with a -dependent coefficient. Our analysis focuses on DSI trained in -space. The benefits of training in a different output space are left as future work. Section 4.2 now claims that DSI is the only parameterisation in which the prior can be used without transformation. The ablations also acknowledge that output transformation is not covered.
Is Figure 2 intended only to compare direct reuse of an endpoint predictor without output transformations? Please distinguish this claim from an advantage over alternatives initialized to preserve the same dynamics through the transformations in Equation (6) and Appendix A.5.
Yes, this is exactly the intent of Figure 2. Appendix A.5.1 shows that output transformation is equivalent to training a DSI with a -weighted loss function. The space in which the loss is calculated is expected to have an effect on the model performance. For instance, in DDPM, Li & He 2026 reports a 0.20 improvement in FID score on ImageNet 64x64 when swapping -loss for -loss in -prediction. However, our experiments and ablations focus on direct SI parameterisations without output transformation. The benefits of implicit learning in different output spaces is left for future work. The manuscript was updated for transparency.
Official response [1/2]
Dear reviewer,
Thank you for your comments. We are glad to hear that you found the spectral diagnostic valuable and share your belief that the approach supports useful ensemble forecasts. Your comments on the noise structure brought forward new perspectives for future work. A new version of the manuscript was uploaded, changes relative to the submission are highlighted in blue.
We agree with your point of view, and this is the motivation for the experiments in Appendix B.8. Theoretically, pure white noise should be sufficient to generate any distribution, including atmospheric states. However, with finite integration steps, imposing different noise structures could make the denoising more efficient by homogenising the signal to noise ratio across spatial scales. Unfortunately, the research on the topic is very limited. Hence, we treat the idea as promising future work.
Nevertheless, we appreciate your idea about short-term conditional uncertainty. We ran an experiment where the white noise is scaled by ERA5 climatological residual uncertainty (STD[xt+Δt−xt]) corresponding to the forecast time (i.e. we compute one variance map per 6h per calendar day). The results were generally negative: Z500 got closer to SSR=1 but had a slight decrease in skill, Q850 became more underdispersive. In general, the large-scale underdispersion was not addressed. Our hypothesis is that this intervention only scales the variance but leaves the spectrum essentially white, which cannot fix the spectral calibration. The current B.8 experiments highlight the importance of spectral properties of the noise, more specifically increasing large scale power and decreasing small scale power (i.e. generally “redder” noise).
A simpler approach to capture short-term uncertainty would be to use ERA5 EDA uncertainty, which quantifies it exactly. We mention this as future work in the paper. Adding a perturbation with variance given by this uncertainty to the initial condition would likely bootstrap large-scale spread and improve the ensemble dispersion on short lead times and beyond.
The manuscript revision of November 11 updates the sampler and subtracts the residual noise at inference. This fixes the noise floor and leaves the variables homogeneously underdispersive. It also includes a spectral postprocessing section that fixes the ensemble calibration across spatial scales.
Figure 15 in Appendix B.1 shows the aggregate SSR for different variables. The lead time evolution is consistent: the ensemble starts underdispersed and gets closer to calibration over the course of a 15-day rollout. The increase is not monotonic; some variables display a small drop in SSR on short lead times. In general, the specific humidity variable (Q) has the best SSR.
The spectral post-processing functions G(ℓ) and H(s) (Figure 29, Appendix B.5) show that the most consequential underdispersion happens in short-term large-scale structures. This is also confirmed by Figure 30 in Appendix B.5 that shows the spectral SSR at different lead times before and after post-processing. Short-term large-scale spread could be increased by including initial condition uncertainty (from ERA5 EDA as in GenCast).