- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PEAR. PEAR is a volumetric transformer for data driven weather forecasting that operates directly on the HEALPix grid. Utilizing an equal area discretization of the sphere allows the model to avoid the polar oversampling issue inherent in standard latitude and longitude grids. The architecture incorporates patch embedding, windowed attention with ring shifting, and hierarchical downsampling and upsampling directly on the one dimensional HEALPix index structure. The authors train and evaluate PEAR on the ERA5 Lite dataset, which is a highly reduced subset of the full ERA5 archive. They compare it against scaled down versions of leading models such as Pangu, GraphCast, FengWu, and AFNO. The submission claims PEAR achieves competitive short term and superior long term forecasting accuracy alongside faster inference times and reduced memory usage. Additional experiments evaluate spectral errors, rotational equivariance, and single step climate model emulation.
Critical strengths
-
The motivation to use equal area grids natively from input to output to solve the polar singularity and oversampling issue is well justified. This approach aligns perfectly with current trends in climate science like the Destination Earth initiative.
-
The architectural adaptations required to make windowed attention work on the one dimensional HEALPix array are elegantly implemented. The authors effectively handle ring shifting and the masking of physically disjoint regions.
-
The paper provides a thorough set of downstream analyses. These include spectral error analysis, rotational equivariance tests, and climate model emulation.
Critical weaknesses
-
The paper trains and evaluates on ERA5 Lite. This dataset consists only of daily samples over 11 years of training data. Modern artificial intelligence weather models derive their predictive power from massive datasets containing hourly or six hourly samples spanning 40 years. Claiming superiority in long term forecasting based on an extremely low data regime is scientifically unsound. Scaling laws in weather forecasting are steep. Performance on ERA5 Lite does not reliably extrapolate to real operational settings.
-
The authors drastically scaled down the baselines to match the exceptionally small parameter count of PEAR. For example, they reduced GraphCast to 4.3 million parameters and Pangu to 11.4 million. The original GraphCast has roughly 36.7 million parameters and uses a level 6 icosahedral mesh. Bottlenecking these architectures to a toy size inherently handicaps designs that were specifically optimized for larger capacities and higher resolutions. This creates a deeply flawed and unequal comparison of model architectures.
-
The core innovation of applying shifted window attention to HEALPix was already established by recent prior work like HEAL SWIN. Extending this concept to three dimensions by adding an explicit vertical dimension and applying it to weather data is an incremental engineering step. It does not represent a fundamental methodological breakthrough suitable for ICLR.
-
ERA5 data is intrinsically generated on a reduced Gaussian grid and distributed on a standard latitude and longitude grid. Projecting this raw data onto HEALPix introduces interpolation artifacts and spatial smoothing. The paper evaluates PEAR on its native HEALPix grid but completely glosses over the information loss incurred during the initial data projection. This omission calls into question the fairness of all native grid metrics.
-
The claims regarding fastest inference and less memory usage are tautological consequences of the experimental setup. The authors compare a highly simplified model against baselines that have been awkwardly squeezed into the same tight parameter budget. These efficiency gains are highly unlikely to hold when the model is scaled up to actual operational resolutions.
-
The climate emulation experiment in Section 5.2 is a single step and fully supervised image translation task. It completely fails to evaluate autoregressive stability or temporal dynamics. Those elements are the core challenges of actual weather forecasting. The inclusion of this experiment feels disjointed and fails to strengthen the main claims regarding forecasting superiority.
Questions for authors
-
How does PEAR perform when trained on the full ERA5 dataset containing 40 years of high frequency samples compared to the original and unscaled versions of GraphCast and Pangu?
-
GraphCast already utilizes an internal representation that shares properties with equal area grids through its icosahedral multimesh. Why is HEALPix fundamentally superior to the icosahedral mesh? This question is especially critical given that your GraphCast baseline was severely restricted to a level 4 mesh instead of its native level 6 mesh.
-
How did you quantify and mitigate the interpolation errors introduced when converting the raw ERA5 data to the HEALPix grid before training began?
-
Could the superior performance at longer lead times simply be an artifact of PEAR having a smoother and more regularized representation? This smoothing from interpolation and a low parameter count might artificially penalize high frequency noise rather than demonstrating a genuinely better understanding of atmospheric dynamics.
2 comments
Dear reviewer,
Thank you again for your time and effort reviewing the manuscript. As the discussion phase is coming to an end, we wanted to check if our responses have addressed your main concerns. Let us know if there are any further clarifications needed.
We thank the reviewer for recognising the motivation for a natively equal-area model and its connection to initiatives such as Destination Earth. We are glad that the reviewer found the ring shifting and masking of physically disjoint regions to be elegantly implemented, and appreciate that the thorough set of downstream analyses was highlighted.
Weaknesses
We have further clarified that we evaluate PEAR and baselines in a controlled setting that is distinct from full-scale ERA5, see lines 23, 280–281 and 481–483 in the updated rebuttal revision (changes in red).
GraphCast-I was chosen from a ten-configuration width and depth sweep by matching PEAR's training-step time. GraphCast-P matches PEAR's parameter count. Our Pangu baseline has 11.4M parameters, 2.6 times larger than PEAR. The mesh level follows the data resolution. The original GraphCast pairs a level-6 mesh with about a million grid points, roughly 25 grid points per mesh node. Our level-4 mesh on 49,000 grid points gives about 19. We have additionally trained GraphCast-P on a level-5 mesh at the same parameter count. It takes 4.3 times the inference time and 3.7 times the peak memory of PEAR, see lines 307–309 and Appendix A.5 (Table A.4). With this mesh GraphCast-P leads PEAR at shorter lead times and PEAR leads at 9 days (PEAR's RMSE is 4.9 % higher at day 1 and 2.2 % lower at day 9).
HEAL-SWIN introduced a shifted-window transformer on HEALPix for spherical vision tasks, and we use its ring shifting along the spherical directions (line 215). The following architectural elements are specific to PEAR (Section 4.2, lines 198–245, Figures B.1 and B.2).
Appendix B.5 (Table B.8) quantifies these: PEAR has 45.6 / 35.5 / 14.4 % lower RMSE at 1 / 5 / 9 days than a variant without shifting, 6.4 / 4.2 / 1.5 % lower than a variant without relative position bias, and 20.4 / 12.9 / 4.9 % lower than a variant with levels as channels.
We agree that we introduce interpolation artifacts when reprojecting the ERA5 data to the HEALPix grid. The 157 × 314 equiangular grid of the baselines is produced from the same 0.25° data by the same bilinear interpolation. This step is common to all models. See lines 290–291 and Appendix C.6 (Table C.9) for a quantitative estimate. To further strengthen the results, we also included results when evaluated on a common equiangular grid, see Appendix C.2 and Tables C.6 and C.7 in the manuscript, where the ranking of models persists.
We compare inference time and peak memory against adapted versions of the baselines that match internal resolution, parameter count or inference cost. See also our response to weakness 1.
We agree that this experiment is not about forecasting and does not test autoregressive stability. We do not claim that this section supports superior forecasting, but rather that it shows the applicability of the model to climate emulation for direct temperature and precipitation prediction from forcing variables.
Questions