- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Tendency-Informed Dynamic Encoding (TIDE) constructs state, first-order difference, and second-order difference features from three consecutive historical states and combines them at multiple scales. Detail-memory branches and gated decoding help reconstruct spatial information. On ERA5 at 0.5° resolution, TIDE outperforms the retrained baselines on several forecast metrics and frequency-band errors.
Critical strengths
Explicit temporal differences are straightforward to implement and integrate into a multiscale network. Using them in both the main pathway and the detail branches provides a practical way to incorporate historical states. The experiments include overall forecast metrics, frequency-band analysis, and module ablations.
Critical weaknesses
The reproduced GraphCast, Pangu-Weather, and FengWu results raise concerns that these baselines were not sufficiently trained or optimized. I do not find the change from 0.25° to 0.5° resolution a sufficient explanation on its own for the apparent performance degradation.
So, the authors should provide training configurations, convergence results, and checks against public reference results under comparable settings. Otherwise, it is difficult to tell whether TIDE’s advantage comes from the method or from weak baseline implementations.
In addition, the evaluation is missing an important reference: climatology. Following WeatherBench2[1], they should also specify how RMSE and ACC are computed and aggregated.
[1] Rasp S, Hoyer S, Merose A, et al. WeatherBench 2: A benchmark for the next generation of data‐driven global weather models[J]. Journal of Advances in Modeling Earth Systems, 2024, 16(6): e2023MS004019.
Questions for authors
-
Can the authors provide the full training configurations and convergence curves for GraphCast, Pangu-Weather, and FengWu, together with reference comparisons under compatible settings, to rule out insufficient training or implementation differences? Can the effect of resolution be tested separately?
-
Can the authors add a climatology baseline on the current 0.5° grid and recheck RMSE and ACC using the WeatherBench2 evaluation procedure, clarifying which variables and lead times retain skill over climatology?
2 comments
I appreciate the authors' effort. Nevertheless, the remaining limitations continue to affect my overall assessment. I maintain my original score.
W1-2, Q1: Unfair Baseline Comparisons
Due to space limitations, the initial version of our manuscript did not provide complete reproduction configurations for the three baselines. We sincerely apologize for this omission. We have now included the relevant details in the appendix of the revised manuscript and added corresponding explanations in the main text. These details demonstrate that all reproduced baselines were adequately trained and had essentially converged.
Furthermore, the original baseline models were trained on high-resolution datasets with a spatial resolution of 0.25°, allowing them to learn from four times as much spatial information as the dataset used in our study. Consequently, their originally reported forecasting accuracy is naturally higher than that achieved by our reproduced implementations. Unfortunately, due to GPU memory limitations, we are unable to fully train these models at 0.25° resolution and therefore cannot conduct comparative evaluations at this resolution.
W3, Q2: Comparison with Climatology
We have additionally evaluated climatology as a baseline using the 2021 test set and updated the corresponding result figures in the revised manuscript. Given the definition of climatology, the ACC metric is not meaningful for this baseline. Therefore, we report only RMSE values in the figures. The RMSE is calculated as follows.
For variable v, forecast initialization time s, and forecast lead time τ, the latitude-weighted spatial RMSE for an individual sample is defined as:
RMSEs,v(τ)=NlatNlon1i=1∑Nlatj=1∑Nlonwi(x^s,τ,v,i,j−xs+τ,v,i,j)2where x^ denotes the predicted value, x represents the corresponding ERA5 ground truth at the forecast valid time, and i and j denote the latitude and longitude grid indices, respectively. The latitude weights are normalized as follows:
wi=Nlat1k=1∑Nlatcosϕkcosϕiwhere ϕi represents the i-th latitude. This normalization ensures that the mean latitude weight is equal to 1.
The RMSE at each forecast lead time is then obtained by taking the arithmetic mean over Nτ forecast initialization samples:
RMSEv(τ)=Nτ1s=1∑NτRMSEs,v(τ)Taking Z500 as an example, the results show that TIDE outperforms climatology for forecast lead times of up to 9 days, whereas Pangu-Weather, GraphCast, and FengWu outperform climatology for lead times of up to 6, 7, and 5 days, respectively.