acceptodds
Withdrawn from ICLR 2027

IMALCast: Intensity-Aware Multimodal Alignment Learning for Heavy-Rainfall Nowcasting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Accurate precipitation nowcasting, especially for extreme rainfall events, plays a vital role in supporting decision-making across multiple sectors of society. However, pointwise regression objectives commonly used in deep learning models are not well aligned with threshold-sensitive operational verification, particularly for high-intensity precipitation regions. Geostationary satellite observations offer complementary cloud information for precipitation forecasting, but differences in spatial and temporal resolution complicate their integration with radar data. To address these challenges, we propose IMALCast, an intensity-aware multimodal alignment learning framework for deterministic precipitation nowcasting. For objective alignment, IMALCast integrates weighted intensity regression with global and high-intensity structural losses motivated by threshold-based verification. For observation alignment, IMALCast uses dual-tower encoding and cross-modal fusion to preserve and integrate modality-specific radar and satellite information. Experiments on the nationwide China Radar–Satellite Dataset and the public SEVIR benchmark demonstrate strong forecasting performance. In particular, IMALCast achieves up to a 65.7% relative improvement in the regional-scale critical success index for heavy rainfall over the strongest baseline in our comparison.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Precipitation nowcasting, spatiotemporal forecasting, intensity-aware learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_AYAvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel framework for extreme rainfall precipitation forecasting. In detail, the proposed method uses a dual-stream encoder for feature extraction from both satellite and radar, and further employs a cross-attention structure to fuse two data source for final rainfall prediction. Experimental results on two public availiable datasets demonstrate the effectiveness of the proposed method.

Critical strengths

None

Critical weaknesses
  1. The authors claimed in the abstract that the fusion of geostationary satellite observation and radar data are very complicated due to the different spitial and temporal resolution. However, the authors didn't explain why fusing satellite and radar are important for precipitation forecasting.
  2. The authors claimed the proposed method is a multimodal learning framework. However, from a deep learning perspective, both satellite and radar observations are represented as gridded spatiotemporal data with similar underlying data structures. Therefore, I believe the authors should further justify the use of the term "multimodal learning" and clarify how their approach differs from conventional multi-source data fusion.
  3. The proposed method lacks novelty. The proposed framework is largely built upon the existing 3D Swin Transformer, and the dual-stream encoder design is also well established in the literature. The main contribution appears to be the modification of the loss function by incorporating evaluation metrics into the training objective. Overall, I didn't find any substantial innovation specifically designed to address the unique challenges of meteorological data.
  4. There is no reference provided for CRSD dataset.
  5. Missing recent baselines [1-2].

[1] Feng, Wenzhi, et al. "Perceptually constrained precipitation nowcasting model." Forty-second International Conference on Machine Learning. 2025.

[2] Perrone Ribeiro, Bernardo, and Jana Faganeli Pucer. "FlowCast: Advancing Precipitation Nowcasting with Conditional Flow Matching." International Conference on Learning Representations. Vol. 2026. 2026.

Questions for authors
  1. The authors should provide a clearer justification for why integrating radar and satellite observations is essential for precipitation forecasting.
  2. The authors need to highlight their contributions.
  3. The authors should explain why radar and satellite belong to different modalities.
  4. The authors should include the missing baselines and discuss their performance.
Reviewer_DyqWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes IMALCast, an intensity-aware multimodal learning framework for heavy-rainfall nowcasting. The method combines intensity-weighted regression with global and high-intensity structural losses to improve threshold-based precipitation detection. It also introduces a dual-tower architecture for integrating radar and satellite observations. Experiments on the China Radar–Satellite Dataset (CRSD) and SEVIR demonstrate improvements in heavy-rainfall detection, particularly at high precipitation thresholds.

Critical strengths
  1. The paper addresses the mismatch between conventional regression objectives and threshold-based precipitation evaluation, particularly for extreme rainfall events. This problem is well motivated and has clear practical significance for meteorological forecasting and early warning systems.
  2. The paper organizes its approach around objective alignment and observation alignment, providing a clear and interpretable framework for heavy-rainfall nowcasting. The proposed structural losses are applied only during training and introduce no additional inference overhead, which is a useful property for practical deployment.
Critical weaknesses
  1. The proposed framework largely combines established techniques, including intensity-weighted regression, cosine similarity losses, dual-tower encoding, and conventional feature fusion. Although the threshold-integrated analysis provides theoretical motivation, the resulting objective remains a relatively straightforward combination of existing losses. The paper does not sufficiently establish what fundamentally distinguishes its optimization strategy from existing threshold-aware objectives or structural losses.
  2. The experimental comparisons rely heavily on relatively old models, including ConvLSTM (2015), TrajGRU (2017), SimVP (2022), and Earthformer (2022). Even the more recent baselines, such as NowcastNet and CasCast, were introduced in 2023 and 2024. For an ICLR 2027 submission, the absence of representative methods from 2025–2026 significantly weakens the experimental evaluation. Moreover, the related work section provides limited coverage of recent developments in precipitation nowcasting. More comprehensive comparisons and discussions are necessary to establish the competitiveness of the proposed approach.
  3. Although multimodal alignment is presented as a central contribution, the performance improvement from incorporating satellite observations is relatively modest. In Table 4, the average CSI increases only from 0.289 to 0.293 compared with the radar-only variant. Furthermore, the SEVIR experiments evaluate only radar inputs. These results provide limited evidence that the proposed multimodal design offers substantial and generalizable advantages.
Questions for authors
  1. Given that the proposed framework combines existing techniques such as weighted regression, structural similarity losses, and dual-tower feature fusion, could the authors further clarify the key methodological differences from prior approaches, particularly FACL and other threshold-aware objectives? What specific aspects of the proposed design constitute a substantial methodological contribution?

  2. Why are representative precipitation nowcasting methods from 2025–2026 largely absent from the experimental comparisons? Could the authors provide additional comparisons with more recent competitive baselines, or explain why the current baselines are sufficient to establish the competitiveness of IMALCast?

  3. Given that incorporating satellite observations only improves average CSI from 0.289 to 0.293 in Table 4, could the authors further explain the practical significance of the proposed multimodal alignment mechanism? How can its generalization capability be established when the SEVIR experiments evaluate only radar inputs?

Reviewer_wRWYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

IMALCast combines weighted MAE with global and soft-thresholded cosine losses for deterministic heavy-rainfall nowcasting. Separate radar and satellite encoders feed a hierarchical spatiotemporal Swin backbone. Experiments cover CRSD and radar-only SEVIR, with objective ablations and transfer to other backbones. The practical results are promising, but evaluation independence and comparisons isolating the proposed objective require stronger support.

Critical strengths
  1. With the backbone and inputs fixed, the proposed loss improves average CSI over FACL and CSI@32/64. Transfer to ConvLSTM, SimVP, and Earthformer supports the usefulness beyond the proposed architecture.
  2. The appendices include three-seed component ablations, parameter sensitivity, paired bootstrap analysis of satellite gains, and distribution-correction experiments.
  3. The problem is clearly defined, and the method is easy to apply.
Critical weaknesses
  1. Event independence and uncertainty need clarification. Training uses 2023–2024, while validation and testing both draw from July–September 2025. The year separation is appropriate, but the validation/test division by storm, date, or time block is unspecified. Evaluation crops are 512×512 with stride 64; adjacent retained crops share 87.5% of their area along one axis. Temporal stride and independent event counts are also missing. Sample-level paired bootstrap may therefore understate uncertainty. Please document event-disjoint splits, separation of input/forecast windows, and event- or block-level confidence intervals. This is an unresolved protocol issue, not evidence that leakage occurred.
  2. Multimodal attribution remains partially confounded (Table 4; B.2; D.3). Radar-only, early-fusion, and dual-tower comparisons already exist, including satellite-gain bootstrap analysis. However, adding satellite input also changes encoder capacity and embedding design. A capacity-matched dual-radar model or shuffled-satellite control would better isolate complementary information from architecture. Radar-only SEVIR cannot independently validate the multimodal contribution.
  3. A closer loss comparison is needed (§3.2; Appendix A; Tables 3–4). The auxiliary objective reduces to asymmetric absolute error up to a target-only constant; it motivates, but does not derive, the cosine losses, as the authors acknowledge. Beyond FACL and component ablations, comparisons with WMAE + soft CSI/IoU or Dice under matched settings would better establish the proposed loss's value.
  4. Clarify aggregation and detection tradeoffs (Tables 1, 5; C.1–C.2). Table 5 implies pooled CSI≈0.118 versus 0.101 in Table 1; different aggregation may explain this, but needs clarification. Higher POD also comes with higher FAR, warranting comparison at matched FAR or POD using validation-selected settings. Generative baselines use ensemble means, which may suppress extremes, so comparisons should be qualified by this protocol. The correctly reported 65.7% gain concerns POOL16; the native-grid gain is 8.6%.
  5. The technical contribution is somewhat limited. This paper seems like an application paper.
Questions for authors
  1. How are validation and test events separated, and do satellite gains persist under event/block bootstrap?
  2. Does the objective outperform matched soft CSI/IoU or Dice alternatives?
  3. How do main-table CSI aggregation and Table 5’s pooled counts correspond, and do gains persist at matched detection tradeoffs?
  4. Does the satellite benefit survive a capacity-matched control?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.