- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies high-resolution regional weather forecasting. It proposes a hybrid model that uses a graph network to read coarse boundary data and convolutional networks to forecast the fine regional grid. The authors train this model as DET-LAM for single forecasts and CRPS-LAM for ensemble forecasts. CRPS-LAM uses a CRPS loss and generates each ensemble member with one model pass per forecast step. On the DANRA dataset, both models outperform the tested graph-based baselines for most variables and lead times.
Critical strengths
- Broad evaluation. The paper uses MEPS and DANRA, tests both ERA5 and IFS boundary inputs on DANRA, and includes checks against station observations. It compares with recent graph-based and diffusion-based methods where they can be run.
- Good skill at a practical cost. With IFS boundaries on DANRA, CRPS-LAM has lower CRPS than Graph-EFM in 164 of 165 variable and lead-time combinations. Table 1 reports about eight minutes for a 20-member, 72-hour CRPS-LAM ensemble on one A100 GPU. These results support the practical value of the method.
Critical weaknesses
- The comparisons do not isolate the benefit of the architecture. Table 1 lists 233 million parameters for DET-LAM versus 23.4 million for Graph-FM, and 251 million for CRPS-LAM versus 14.5 million for Graph-EFM. Graph-FM uses an 800 km boundary region without overlap, while the proposed models use an overlapping 400 km region. Similar inference time is a useful practical comparison, but it does not show how much of the accuracy gain comes from the hybrid design. A smaller version of the proposed model and a comparison with matched boundary inputs would help.
- The ensemble fields need a more direct check. Could you test the spatial and temporal structure of individual ensemble members on the DANRA test set? For example, please report the CRPS of 3-hour changes along each member’s forecast trajectory, at several lead times. A comparison with Graph-EFM would show whether CRPS-LAM improves ensemble quality beyond pointwise scores.
Questions for authors
- Could you compare smaller DET-LAM and CRPS-LAM models with the graph baselines at similar parameter counts? Could you also compare DET-LAM and Graph-FM with the same boundary region? This would help separate the effects of model size, boundary inputs, and architecture.
- Could you measure whether ensemble members preserve realistic spatial and temporal links? For example, compare correlations between nearby locations, changes along a forecast trajectory, or a spatial score.
- Section 3 says that using 32 dimensions for edge features saves memory without hurting performance. Could you show an ablation with larger or smaller edge dimensions, reporting both forecast scores and peak memory use?
- Could you show how CRPS-LAM’s forecast skill change beyond 72 hours (for example, at 5–7 days) using IFS or ERA5 boundary forecasts? I think [Why we did not scale further] is the reason for diffusion-based models, rather than for CRPS-LAM.
- Figure 21 should use different colors for DET-LAM and CRPS-LAM, and the unresolved reference near line 400 should be fixed.
We thank the reviewer for the thorough review and the detailed suggestions. We are glad that the reviewer appreciated the broad evaluation, covering MEPS and DANRA, both ERA5 and IFS boundaries and station observations, and found that the forecast skill of CRPS-LAM at a practical cost supports the value of the method. Below we address each weakness and question.
Q1/W1: Model size. We do not claim that the hybrid is better than a GNN at equal parameter count, but that it is much more efficient on current GPUs and software, which lets us use ten times the parameters at a lower inference cost: predicting each time step with DET-LAM (233M) is 30% faster than with Graph-FM and uses less memory. To separate the effect of model size from that of the architecture, we have started training parameter-matched variants DET-LAM-S and CRPS-LAM-S.
Boundary region. Our Graph-FM baseline follows [1], where the overlapping boundary configuration introduced unphysical oscillations and the non-overlapping configuration was therefore recommended. We agree that a comparison with matched boundary inputs adds value, and have started training Graph-FM with the same overlapping 400 km boundary region.
Training even the smaller models requires an estimated 1,500–3,000 GPU hours per run, so we cannot promise results within the discussion period, but we will report all runs that finish in time.
[1] Adamov et al. Building machine learning limited area models: Kilometer-scale weather forecasting in realistic settings. arXiv:2504.09340, 2025.
Q2/W2: We have added an evaluation of the spatial structure of the ensemble members in the revised version (Appendix G). We report the CRPS of average- and max-pooled fields (Appendix G.1), and variogram and energy scores on local patches (Appendix G.2). For the links between nearby locations, we report a distance-weighted variogram score over pairs of grid points up to 100 km apart (Appendix G.2).
CRPS-LAM outperforms Graph-EFM in this evaluation. The variogram and energy scores are lower for all 14 evaluated variables, with a single exception at 72 h. The improvement in pooled CRPS grows when the fields are averaged over larger regions.
Q3: We thank the reviewer for this suggestion. We agree that this claim is currently not justified by any experiments. Our choice of 32 edge feature dimensions was motivated by the findings in [1], where the authors investigate the effect of reducing the edge feature dimensionality and report no noticeable degradation in predictive performance. We adopted this design choice to reduce the memory requirements of our models. In our revision we have updated our claim to "This results in a significant reduction in memory cost. Based on results in the literature for similar settings [1], we do not expect this lower edge dimensionality to impact the performance significantly, and believe that the saved memory is better used for other model components." (line 194).
We agree that an ablation study varying the edge feature dimensionality, with both forecast performance and peak memory usage reported, could provide a more comprehensive evaluation. Unfortunately, we are not certain that we will be able to conduct an additional ablation within the discussion period. Training a single model requires approximately 3,500–7,000 GPU hours, depending on the model configuration, and the available computational resources and limited time do not permit the additional training and evaluation runs required for a meaningful comparison. Furthermore, larger edge feature dimensions lead to out-of-memory errors for our largest model configurations on the available GPUs, preventing us from properly evaluating these settings. We do not expect that such an evaluation would lead to a different conclusion than in [1].
[1] Alet et al. Skillful joint probabilistic weather forecasting from marginals. arXiv:2506.10772, 2025.
Q4: The paragraph "Why we did not scale further" refers to the size of the diffusion baseline, i.e. why we did not train a larger denoiser on DANRA, and not to the forecast length of CRPS-LAM.
The 72 h horizon follows from the limited-area setting and is not a computational limitation of CRPS-LAM. At 5–7 days the information in the high-resolution initial state has largely left the domain, so the forecast is mainly determined by the global forcing entering through the boundary, and an evaluation would mostly measure the skill of that forcing. In addition, CRPS-LAM is fine-tuned with rollouts of up to 36 h and calibration degrades beyond the training rollout length (new Appendix F), so a meaningful 5–7 day evaluation would probably also require fine-tuning on longer rollouts, which we cannot complete within the discussion period.
Q5: Thank you for noticing this. We have fixed the unresolved reference and changed the color of DET-LAM in the plots.