- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces EarthAlign, a dataset of 553,738 samples pairing 63-variable ERA5 data with four styles of natural-language descriptions. It also proposes MeteoLIP, a framework for aligning meteorological and textual representations, with MetTok incorporating temporal and geographic information into variable aggregation. Experiments on cross-modal retrieval, weather-condition classification, and physical variable estimation show improvements over the corresponding CLIP and SigLIP baselines.
Critical strengths
EarthAlign addresses a meaningful need for large-scale meteorology–language data. Its combination of diverse event records, normal-weather samples, and multiple caption styles provides a useful resource for future research.
Critical weaknesses
-
The differences between MetTok and ClimaX’s variable aggregation design need clarification.
-
The WeatherQA experiment lacks a controlled comparison to isolate the benefit of EarthAlign pretraining for downstream transfer.
Questions for authors
-
ClimaX [1] already encodes variables separately and aggregates their embeddings through cross-attention with a learnable query. Could the authors clarify how MetTok differs from this design and what additional benefits it provides?
-
The authors have analyzed retrieval performance after removing temporal information from the captions. How does the model perform when candidate descriptions refer to similar times and nearby locations, so that distinguishing them requires meteorological information? Could the authors provide an evaluation under this setting to further clarify the contribution of meteorological content to retrieval?
-
Appendix F.4 reports the transfer results of Meteo-CLIP on WeatherQA. Could the authors compare it with an encoder without EarthAlign pretraining while keeping the downstream language model and training settings unchanged? This would help establish the transfer benefits of EarthAlign pretraining.
[1] Nguyen, T., Brandstetter, J., Kapoor, A., et al. ClimaX: A Foundation Model for Weather and Climate. arXiv preprint arXiv:2301.10343, 2023.
4 comments
Response to Reviewer 46Z6 (2/2)
Weakness 2 & Question 3: Benefit of EarthAlign pretraining on WeatherQA
Following your suggestion, we have now added this comparison to Appendix F.7. The baseline uses the same encoder architecture as Meteo-CLIP but without EarthAlign pretraining. The baseline ViT is initialized from OpenAI CLIP and its MetTok is randomly initialized, while the downstream language model and its training settings are kept the same. Since the baseline encoder has never been trained on meteorological data, we also fine-tune the ViT and MetTok end-to-end on WeatherQA, whereas Meteo-CLIP remains frozen. This should favor the baseline to some extent.
| Encoder | Encoder on WeatherQA | Area Affected (%) | Concerning (%) |
|---|---|---|---|
| Without EarthAlign pretraining | fine-tuned end-to-end | 34.91 ± 0.57 | 48.86 ± 1.72 |
| Meteo-CLIP (EarthAlign-pretrained) | frozen | 49.22 ± 4.45 | 51.73 ± 2.41 |
Despite being frozen, Meteo-CLIP achieves higher accuracy than the baseline on Area Affected (+14.31 points) and Concerning (+2.87 points). This supports the transfer benefit of EarthAlign pretraining to downstream severe weather reasoning.
Thank you for your detailed response. My concerns have been addressed, and I will maintain my current positive rating.
Thank you for your thoughtful review and for confirming that our clarifications and additional results have addressed your concerns. We also appreciate your constructive suggestions, which have helped us strengthen the work.
Response to Reviewer 46Z6 (1/2)
We thank the reviewer for the constructive review and for recognizing the need for large-scale meteorology-language data that EarthAlign addresses. Below we address (1) the differences between MetTok and ClimaX (W1 & Q1; Sec. 4.1), (2) retrieval with candidates from similar times and nearby locations (Q2; Appendix F.4), and (3) a controlled WeatherQA comparison (W2 & Q3; Appendix F.7). In addition, we have uploaded a revised manuscript with all changes highlighted in gold.
Weakness 1 & Question 1: Differences between MetTok and ClimaX's variable aggregation
We agree that MetTok shares some similarities with ClimaX's variable aggregation. However, since ClimaX is designed for weather forecasting, MetTok differs from it in three aspects that suit meteorology-language alignment:
These aforementioned differences arise from our different objective and inputs. We designed MetTok for meteorology-language alignment rather than forecasting, with regional inputs that vary in location and extent. We have now credited ClimaX and briefly stated these differences in Sec. 4.1 of the updated manuscript.
Question 2: Retrieval when candidates refer to similar times and nearby locations.
Based on your suggestion, we have now added an additional retrieval evaluation in Appendix F.4 in the revised manuscript. The evaluation is built on the time-removed captions, and thus the time cue in the descriptions is removed. For each query, the candidate pool contains only test samples within ±72h of the query (i.e. one synoptic period) whose ERA5 crop centers lie within 50 km of the query's, so all candidates refer to a nearby period and location. Here we use the improved time-removed retrieval benchmark of the revised manuscript (Appendix F.3). Following Reviewer iuG2's suggestion, we now identify for every sample all captions that correctly describe it, i.e. captions whose information is entirely contained in the sample's metadata record, and vice versa. These matches are obtained with an LLM judge followed by rule-based consistency checks, so that equally valid matches are no longer counted as errors. Within a pool, all valid matches of a query count as a single candidate, and only queries with at least four incorrect candidates are kept. This yields 1,335 queries with 8.3 candidates per pool on average. In addition, we report random guessing as a reference.
In this setting, Meteo-CLIP is 15.6 (M2T) and 19.9 (T2M) points above random guessing, whereas the metadata-only model is only 7.9 and 15.0 points above random guessing. Meteo-CLIP roughly doubles the gain over random guessing in M2T (15.6 vs. 7.9) and increases it by about one third in T2M (19.9 vs. 15.0). Since the candidates come from similar times and nearby locations and their captions contain no time information, this additional gain further shows the importance of the meteorological data for distinguishing them.