- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3(+1)
- Soundness
- 2
Read review
Paper summary
The paper introduces RainAtlas, a dataset and benchmark for precipitation downscaling and cross-regional generalization. It harmonizes hourly precipitation observations from the United States, Europe, and East Asia onto 2 km grids and pairs them with ERA5 and static geographical covariates. Comparisons of deterministic and generative methods show favorable performance for generative approaches across several metrics, while cross-regional generalization depends strongly on the training region.
Critical strengths
-
The dataset has clear practical value. Pairing heterogeneous precipitation products across multiple regions with standardized input variables reduces the data preparation burden and provides a reusable foundation for cross-regional downscaling research.
-
The comparison includes deterministic and several generative methods, with multiscale and probabilistic evaluation under both single-region and multi-region training. So, I believe that the benchmark offers broad experimental coverage.
Critical weaknesses
-
Nearest-neighbor resampling and the removal of entire time steps based on anomalous values may affect precipitation totals, extremes, and spatial structures, but the main text lacks quantitative before-and-after comparisons. For a dataset contribution, these choices should be validated for their effects on key precipitation statistics and model comparisons.
-
Regions differ simultaneously in observational products and training periods, so the reported generalization gaps reflect several distribution shifts. More targeted, matched comparisons are needed to distinguish the roles of climatic diversity, observational quality, and temporal coverage. Therefore, the interpretation of cross-regional performance remains under-supported.
Questions for authors
-
Does the cross-regional advantage of MRMS-trained models persist after matching sample counts and seasonal distributions and stratifying by climate regime and observational quality?
-
Can representative precipitation fields and error maps support this analysis?
3 comments
W2. Several distribution shifts at once
"Regions differ simultaneously in observational products and training periods, so the reported generalization gaps reflect several distribution shifts. More targeted, matched comparisons are needed to distinguish the roles of climatic diversity, observational quality, and temporal coverage."
On observational quality: a matched comparison would require a second product over the same region and period. Such products exist for parts of the US (NCEP Stage IV) and Europe (RADKLIM), but they are either coarser or built from the same radar measurements. RainAtlas provides a common framework in which such products could be added. We will clearly state this in the limitations, as an avenue for future work.
On temporal coverage: the validation (2021) and test (2022) years are the same for all three regions. Only the training windows differ, by 5 to 6 years, between MRMS and EURADCLIM / EA. Over that offset, the forced change in the precipitation distribution is negligible compared with internal climate variability (details in our response to Reviewer YDwa, concern D). How this variability affects model performance remains an open question, which we will state as future work.
On climatic diversity: we considered using the sampling strategy to test whether over-representing certain Köppen-Geiger classes improves geographical generalization, but decided it was outside the scope of this paper. We will explicitly state this direction as future work.
We still think the reader deserves hypotheses for the MRMS result. The two we find most plausible are that (1) MRMS is the most homogeneous product, built from the largest number of ground-based radars in a single unified network, and (2) it covers the most climatically diverse domain. The revised text will present these as hypotheses that the paper's results do not test, and as an invitation for further work on RainAtlas.
Q1. Does the MRMS advantage persist after matching?
"Does the cross-regional advantage of MRMS-trained models persist after matching sample counts and seasonal distributions and stratifying by climate regime and observational quality?"
Sample counts are matched by construction: every test set contains 30,000 crops and every training set 150,000 crops, for every region and combination of regions. We refer the reviewer to our response to W2 regarding stratification by climate regime and observational quality.
Q2. Representative fields and error maps
"Can representative precipitation fields and error maps support this analysis?"
Spatial error maps would be a valuable complement, but they cannot be derived from our current evaluation. Our test sets consist of 30,000 crops sampled across space and time with importance or random sampling, so they do not cover each domain completely. Per-grid-cell error maps would require new test sets spanning each full domain with overlapping crops over a fixed period, and inference for every model and training configuration, at a substantial additional compute cost. We therefore leave domain-wide error maps to future work.
Regarding precipitation fields, we thank the reviewer for the suggestion. In the camera-ready version, we will add visualizations of the predicted precipitation fields from all benchmarked models and training configurations for four intense precipitation events per domain, selected from the test set using an objective criterion (e.g., the highest domain-maximum intensities). This will let readers directly compare how each model reproduces extreme precipitation structures, both in-domain and on held-out regions.
We thank the reviewer again for their review. We believe we have answered each of the reviewer's concerns and questions, and we would welcome further discussion if any remain unaddressed.
Thank you for your thorough and patient responses. My concerns have been addressed, and I believe this work meets the standards for acceptance at ICLR. I increased my rating. Good luck!
We thank the reviewer for the review, and for noting the "clear practical value" of the dataset and the "broad experimental coverage" of the benchmark. We address the weaknesses and answer the questions below and in the following comment.
W1. Effect of nearest-neighbour resampling and timestep removal
Schemes such as bilinear interpolation or conservative (area-weighted) remapping average the ~4 source cells that fall into each 2 km cell, which smooths the field and lowers peak intensities. Nearest-neighbour keeps one source value per target cell unchanged, so observed intensities are never smoothed, at the cost of occasionally dropping source values, sometimes including the local maximum. We quantified the impact of each scheme on MRMS over 2022, on 500 random hours and on the most intense hour of the 100 most intense days, comparing the native 0.01° field (area-weighted statistics) with the released nearest-neighbour regridding and with bilinear and conservative remapping onto the same 2 km grid:
Nearest-neighbour preserves the intensity distribution (wet fraction, mean and upper quantiles within 0.1 mm/h of the native values), whereas bilinear and conservative remapping slightly increase light rain (+3% wet fraction) and lower the tail statistics. We agree that this should be documented, and will add this table to Appendix A in a new paragraph on reprojection and resampling. The EA dataset is resampled from a 0.01° grid with the same nearest-neighbour scheme, so the same conclusions hold there.
Regarding the removal of entire timesteps from EURADCLIM, we quantified its effect on the resulting distribution. The filter leaves the wet fraction unchanged and slightly lowers the upper quantiles, with the largest effect at the 99.99th percentile (16.8 → 14.9 mm/h). Quantiles are computed over all cells of the timesteps considered, dry cells included. We thank the reviewer for this recommendation and will include this table in Appendix A.