acceptodds
Under review as a conference paper at ICLR 2027

STCFormer: Adaptive Spatio-Temporal Modeling with Dynamic Cluster Transformer for Station-based Weather Forecasting

Abstract

Station-based weather forecasting supports daily life and economic activity, yet accurate forecasts require modeling complex spatial dependencies among stations. Recent clustering-based selective modeling offers a promising alternative to dense inter-station interactions. However, a grouping shared across an observation window may obscure local changes in station relationships, while intra-cluster interactions alone may miss important global context. The theoretical advantages of selective interactions over dense connectivity also remain insufficiently understood. We therefore propose STCFormer, an adaptive spatio-temporal Transformer that dynamically groups stations according to their local evolution within each temporal patch. Its Cluster-Guided Attention Block combines fine-grained local attention within clusters and global attention over regional state summaries, allowing each station to access information beyond its own cluster. We further show that a derived Lipschitz upper bound for cluster-conditioned local attention is no larger than its fully connected counterpart, explaining a potential robustness benefit and motivating the design of InfoLoss. Experiments on three real-world weather datasets spanning eight temperature and wind forecasting tasks show that STCFormer achieves the lowest 24-hour mean squared error on all eight tasks and ranks first or second in 47 of 48 comparisons across metrics and forecasting horizons. Ablations and case studies further confirm the benefits of locally adaptive grouping and complementary local–global interactions.

Primary area:
learning on time series and dynamical systems
Keywords:
Station-based Weather Forecasting, Clustering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Z6CcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose STCFormer, an adaptive spatial-temporal Transformer for station-based weather forecasting. The Cluster-Guided Attention Block (CGAB) first performs hard-routed local attention within clusters, then aggregates the updated state features into regional content centers. Global station-to-region attention allows each station to access summaries from all regions, combining selective local interactions with global context. Experiments on three real-world weather datasets support the proposed design.

Critical strengths
  1. The paper is clearly written.
  2. The paper demonstrates the effectiveness of the proposed method through both theoretical analysis and empirical experiments.
Critical weaknesses
  1. The motivation and technical innovations of the proposed method are not inherently tied to the weather forecasting application. Clustering related stations is a general approach that can be applied to any multivariate forecasting task. However, clustering-based techniques for improving multivariate time series forecasting have already been extensively studied (e.g., Channel Clustering and DUET), and the paper does not present any application-specific innovations that distinguish its approach in the context of weather forecasting.
  2. Given the recent advances in weather forecasting models pretrained at scale on reanalysis data, I have a fundamental concern about the practical value of forecasting models that still require end-to-end training for specific weather stations. For example, in the GlobalTemp dataset, it is difficult for a model to effectively learn complex global spatial dependencies from 3,850 sparsely distributed stations.

Minor issues

  • \citep should be used instead of \cite to ensure proper citation formatting.
  • In Section 2.3.1, the paragraph of “M-Step” should be moved to a new line.
Questions for authors

Existing station-based weather forecasting models typically rely on end-to-end training for specific stations, which significantly limits their practical applicability. Can the proposed method support zero-shot inference on unseen stations that are similar to those within a cluster?

4 comments
AuthorsOpenReview ↗

Respond to W1: What weather-specific contributions and insights distinguish STCFormer from existing channel-clustering methods such as DUET?

We thank the reviewer for the suggestion. Channel clustering techniques, such as DUET, have been extensively studied in multivariate time series forecasting. We would like to clarify that we do not simply transfer these techniques to weather forecasting. Compared with these methods, we offer the following additional contributions and insights tailored to the characteristics of station-based weather forecasting.

1) Patch-level clustering for the dynamic evolution of weather systems. The propagation and evolution of weather systems cause station relationships to change within the observation window. Window-level shared clustering in methods such as DUET can easily obscure these local changes. We therefore propose a dynamic clustering mechanism that updates at each temporal patch, allowing the interaction scope to adapt to the local evolution features of stations.

2) Representation design incorporating station geography and evolution features. Considering the local correlations among geographically neighboring stations, we are the first to construct patches along the spatial dimension to preserve local spatial context. Meanwhile, the dual state-evolution views provide temporal context for prediction and the basis for dynamic grouping, respectively, allowing clustering to respond to local changes in weather processes.

3) Zero-shot inference on new stations. As discussed in the response to W2 below, newly established stations and stations with limited training data have practical forecasting needs. Our method supports zero-shot inference on new stations that were not involved in training, extending its applicability to changing station networks. This cannot be achieved by centroid-free methods such as DUET.

4) Theoretical justification for clustering-constrained attention. For selective interactions among stations, we prove for the first time that the derived Lipschitz upper bound of intra-cluster local attention under a given partition is no larger than the corresponding upper bound of fully connected attention, establishing its theoretical superiority.

Taken together, these contributions and insights distinguish our method from related channel-clustering work in the weather forecasting setting.

AuthorsOpenReview ↗

Respond to W2/Q: What is the practical value of station-based forecasting, and can STCFormer perform zero-shot inference on unseen stations?

We thank the reviewer for the suggestion. We fully acknowledge the success of recent large-scale pretrained weather forecasting models. At present, these models primarily focus on gridded forecasting, while pretrained models available for station-based forecasting are often general-purpose time series foundation models. Many of these models adopt channel-independent modeling, making it difficult to fully use the spatial interactions that are essential for station-based forecasting and thereby limiting their applicability in this setting. The experiments in Appendix I also demonstrate the forecasting advantages of our method over the compared time series foundation models.

We also thank the reviewer for the insight regarding generalization to new stations. Our method is structurally well suited to zero-shot inference. On the one hand, learnable prototypes acquire spatial association patterns among stations from the training data, providing a reference for grouping and information exchange at new stations. This is absent from methods such as DUET. On the other hand, the model architecture and dynamic clustering mechanism do not depend on a fixed number of stations and can flexibly include new stations that were not involved in training in the clustering process. The table below presents zero-shot inference results for forecasting the next 72 hours on the France dataset. Specifically, we randomly select 210 stations for training and validation and evaluate on the remaining 24 stations. We compare our method with the time series foundation model Timer and DLinear trained specifically on these 24 stations. The results show that our method continues to achieve a consistent lead, supporting its zero-shot generalization ability on unseen stations. We will supplement the revised manuscript with the complete relevant experiments and discussion.

ModelTemp.U-WindV-Wind
Timer7.415 ± 0.0124.932 ± 0.0075.813 ± 0.020
Dlinear6.928 ± 0.0074.675 ± 0.0124.727 ± 0.003
STCFormer (Zero-Shot)6.315 ± 0.0264.385 ± 0.0364.659 ± 0.138
AuthorsOpenReview ↗

Respond to Minor issues

We thank the reviewer for the suggestions. We have addressed the citation and paragraph formatting issues raised by the reviewer in the paper.

Thanks you for your response. My main concerns remain unresolved:

  1. The paper does not present enough innovations that are strongly tailored to the meteorological domain.
  2. Second, the authors fail to address the practical significance of station-level forecasting comparing to reanalysis-based models. Moreover, even when comparing against time series foundation models, the evaluation does not include state-of-the-art models such as Chronos-2 and TimesFM-3.

Therefore, I decided to keep my original score.

Reviewer_PJe3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes STCFormer, an adaptive spatio-temporal Transformer for station-based weather forecasting, motivated by the tension between window-shared grouping, which may obscure locally changing station relationships, and purely intra-cluster interaction, which may lose useful global information. STCFormer dynamically updates station groups at the temporal-patch level according to local evolution, and combines intra-cluster local attention with global station-to-region attention through its Cluster-Guided Attention Block. The paper further establishes a theoretical connection among mutual information, maximum cluster size, and an upper bound on the sensitivity of local attention, and provides perturbation experiments consistent with the theoretical ordering. Experiments are conducted on eight temperature and wind forecasting tasks from three real-world weather datasets, with comparisons against spatio-temporal and multivariate time-series models.

Critical strengths

1. The motivation is clear, and the dynamic grouping and local–global interaction mechanisms are well aligned with the identified problem. The paper explicitly identifies the conflict between window-shared grouping and locally evolving station dependencies, while also considering the information loss caused by restricting interactions purely within clusters. The resulting patch-level adaptive grouping and cluster-guided local–global attention provide a coherent response to these two issues.

2. The theoretical and empirical analyses are relatively comprehensive. The paper develops a theoretical chain connecting mutual information, maximum cluster size, and an upper bound on local-attention sensitivity, and the fixed-partition perturbation experiments support the corresponding empirical sensitivity ordering. The experimental comparison also covers a broad range of spatio-temporal and multivariate time-series baselines.

Critical weaknesses

1. The Related Work does not sufficiently position the proposed grouping granularity within existing dynamic dependency modeling approaches. The paper mainly starts from window-shared grouping and motivates patch-level adaptive grouping, but does not clearly distinguish window-level, patch-level, and the step-level dynamic dependency modeling commonly adopted in spatio-temporal graph methods. Consequently, the position of the proposed clustering-based patch-level design within the broader spectrum of dynamic dependency modeling remains insufficiently articulated.

2. The theory emphasizes the effect of grouping quality on the downstream operator, while the model’s adaptivity is still constrained by predefined spatio-temporal patches. Before dynamic clustering, both the temporal and station axes are partitioned into fixed spatio-temporal patches determined by LtL_t and LsL_s, and the subsequent clustering operates on these spatial patch tokens rather than on the original stations. Although cluster assignments are recomputed across temporal patches and encoder layers, the model cannot dynamically modify the temporal patch boundaries or the composition of original stations within each spatial patch. Meanwhile, the perturbation experiments condition on a fixed realized partition. This leaves a clear gap between the theoretical emphasis on grouping quality and the range over which the implemented model can actually adapt its grouping.

3. The baseline categories are broad, but recent strong methods within these categories are not sufficiently covered. The experiments already include spatio-temporal models, multivariate time-series models, and additional grid-based predictive models in the appendix, but the comparisons within these categories are not fully updated to recent strong methods. For example, PhysicsFormer has appeared after S2 Transformer in global station weather forecasting, while recent 2026 MTS forecasting methods such as ConFlux, FACT and VPNet study fine-grained dynamic cross-variate interactions and scalable local cross-variate dependency modeling, respectively, and report strong forecasting results. Therefore, the current experiments demonstrate competitiveness against a diverse benchmark set, but provide weaker evidence relative to the current state of the art within each category.

Questions for authors

1. Comparison with large-graph and dynamic-relation modeling approaches. Have the authors considered comparisons with representative spatio-temporal graph methods designed for large graphs, such as MGCN, or methods motivated by dynamically changing dependencies, such as DMSTGCN? If not, what was the main reason for excluding such comparisons? This is mainly to better understand how clustering-based interaction differs from alternative dynamic-graph modeling strategies.

2. Fixed spatio-temporal patches and dynamic grouping. Since dynamic clustering is performed on predefined spatio-temporal patch tokens rather than directly on the original station-time observations, how do the authors interpret cases where station relationships change within a temporal patch, or where stations grouped into the same spatial patch exhibit different dependency evolution? To what extent does the fixed patch construction limit the adaptivity of the subsequent dynamic grouping?

3. Fixed-partition assumption in the theoretical analysis. The current sensitivity analysis conditions on a fixed realized cluster partition, and the perturbation experiment likewise freezes the partition before perturbing the input. If cluster assignments are recomputed after perturbation, does the empirical sensitivity ordering remain similar? How do the authors expect assignment changes themselves to affect the connection between the theoretical bound and the behavior of the actual dynamic grouping mechanism?

5 comments
AuthorsOpenReview ↗

Respond to Q1/W3: How does STCFormer compare with recent strong baselines and large-scale graph methods, and how does its interaction mechanism differ?

We thank the reviewer for the suggestion. We fully agree that further comparisons should be made with recent strong methods in each category, as well as methods for large-scale graphs and dynamic dependency modeling. The table below presents the results of our method, MGCN [1], and FACT [2] for forecasting the next 24 hours on the France dataset. As can be seen, our model continues to achieve consistently superior performance.

ModelTemp.U-WindV-Wind
MGCN6.948 ± 0.0854.064 ± 0.0684.926 ± 0.024
FACT6.302 ± 0.0314.051 ± 0.0654.901 ± 0.026
STCFormer (Ours)6.291 ± 0.0904.045 ± 0.0284.883 ± 0.013

Regarding the reviewer's question about the differences between clustering-based interactions and other dynamic graph modeling strategies, the key lies in how they adapt to actual weather conditions during testing. For methods whose graph structure is fixed after training, the interaction structure has difficulty fully responding to the continuously changing meteorological relationships in test samples. Our dynamic clustering mechanism instead regroups stations during testing based on the local evolution features of the current input, flexibly constructing an intra-cluster attention graph. We will subsequently supplement the comparison results with these methods on all datasets.

[1] Wenxie Lin, Zhe Zhang, Gang Ren, Yangzhen Zhao, Jingfeng Ma, and Qi Cao. 2025. MGCN: Mamba-integrated spatiotemporal graph convolutional network for long-term traffic forecasting. Knowledge-Based Systems, 309, 112875.

[2] Huiqiang Wang, Jieming Shi, and Qing Li. 2026. FACT: Fine-grained Across-variable Convolution for Multivariate Time Series Forecasting. In International Conference on Learning Representations (ICLR).

AuthorsOpenReview ↗

Respond to Q2/W2: How do predefined spatial patches affect the adaptivity of the subsequent dynamic grouping mechanism?

We thank the reviewer for the suggestion. On the one hand, larger spatial patches incorporate broader local spatial context, improve computational efficiency, and help tighten the sensitivity upper bound in the theoretical analysis. On the other hand, as the reviewer pointed out, they may also limit fine-grained adjustments to changes in station relationships within a patch. We therefore view this as a trade-off among modeling granularity, broader spatial context, and computational efficiency. The spatial patch-size sensitivity experiments in Appendix I.3.2 show that, on the evaluated datasets, appropriately sized spatial patches consistently outperform the setting with a patch size of 1. From the perspective of forecasting performance, this indicates that moderate spatial aggregation can achieve an effective balance between fine-grained adaptivity and broader spatial context, providing useful local representations for subsequent adaptive modeling and thereby improving forecasting performance.

AuthorsOpenReview ↗

Respond to Q3/W2: Does recomputing cluster assignments after perturbation change the empirical sensitivity values, their ordering, and the theoretical interpretation?

We thank the reviewer for the suggestion. We first provide below the empirical sensitivity bound values and their ordering when cluster assignments are recomputed after perturbation. The values change only slightly, while the sensitivity ordering remains consistent. This is because Theorem 2.1 in the paper holds for any given fixed partition. In essence, the theorem characterizes the theoretical advantage of block-diagonal local attention over global attention. This advantage is mainly determined by the maximum cluster size. Even if the specific cluster partition changes, as long as the maximum cluster size does not change significantly, changes in cluster assignments and dynamic clustering will not dramatically affect the theoretical bound.

SettingComputed before perturbationComputed after perturbation
Dense3.8256 (5)3.8256 (5)
Imbalanced3.1824 (4)3.2574 (4)
Random2.9542 (3)2.9621 (3)
Learned2.7020 (2)2.7133 (2)
Balanced2.6266 (1)2.6282 (1)
Reviewer_jAkXOpenReview ↗
Rating
1
Confidence
4(+1)
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents STCFormer, an adaptive spatio-temporal Transformer for station-based weather forecasting. It forms joint spatio-temporal patches, constructs state and local-evolution views, dynamically clusters station tokens at each temporal patch, and applies a Cluster-Guided Attention Block that combines hard-routed intra-cluster attention with station-to-region global attention. A fixed-partition Lipschitz analysis motivates the InfoLoss regularizer.

Critical strengths

The combination of patch-specific grouping, fine-grained intra-cluster attention, and a global station-to-region path is coherent because it allows selective local interaction without completely isolating different groups. The empirical evaluation is broad and well structured. It covers three real-world datasets, eight forecasting tasks, multiple horizons and metrics, 15 baselines, three-run uncertainty, and targeted ablations of window-shared grouping, static grouping, InfoLoss, and the local and global attention branches; these results consistently support the practical value of the proposed design.

Critical weaknesses

There is a scope mismatch between the theoretical analysis and the full model. The analysis assumes a given station partition and considers only intra-cluster local attention, whereas the actual partition is input-dependent and CGAB also includes cross-cluster station-to-region attention. The results therefore provide useful motivation for the local branch, but should not be interpreted as a theoretical guarantee for the complete STCFormer. The dynamic grouping mechanism has limited explicit modeling of temporally propagating relationships. Clustering is based mainly on the current patch representation and its first-order change, which favors synchronously evolving stations and does not explicitly account for lagged or directional dependencies induced by propagating weather systems. Moreover, cluster centers are recomputed at each temporal patch without being carried forward, so transitions and longer-term evolution of the grouping states are not modeled explicitly.

Questions for authors

See Weaknesses.

2 comments
AuthorsOpenReview ↗

Respond to W1: What is the scope of the theoretical analysis, and how does it relate to the complete STCFormer model?

We thank the reviewer for the suggestion. We would like to clarify that the theoretical analysis in this paper has consistently focused on the advantage of clustering-constrained local attention over fully connected station attention in terms of the derived Lipschitz upper bound. This analysis is a core theoretical contribution of our paper and provides a design motivation for adopting clustering-constrained local attention. Although the CGAB module in the paper also includes global attention, our conclusions have consistently been restricted to the local-attention operator, without overclaiming an overall advantage or a theoretical guarantee for CGAB or the complete STCFormer. We have strengthened the relevant statements in the paper to prevent misunderstanding.

AuthorsOpenReview ↗

Respond to W2: How does dynamic grouping capture propagating dependencies and the longer-term evolution of grouping states?

We thank the reviewer for the suggestion. First, we would like to clarify that clustering at each temporal patch combines features from the current and preceding patches to group stations with similar local evolution. As the cluster partition is updated patch by patch, changes in station relationships during the propagation of weather systems can be reflected in the dynamic interaction structure. Meanwhile, temporal attention integrates context from the entire observation window, allowing the model to use historical information across patches and model dependencies over longer time spans.

In addition, regarding the suggestion to explicitly model transitions in grouping states, we explored this during the early stages of the project. In early experiments, we introduced a regularization term that maximizes the mutual information between cluster partitions of adjacent temporal patches, as shown below:

Rtemp=−1T′−1∑p=2T′I!(Yp−1;Yp).R_{temp}=-\frac{1}{T'-1}\sum_{p=2}^{T'}I\\!\left(Y_{p-1};Y_p\right).

This term is intended to explicitly strengthen the statistical correlation between adjacent partitions and indirectly influence centroid estimation for subsequent patches during training. The table below presents comparative results for 24-hour forecasting on the France dataset. Adding this term did not produce consistent improvements in forecasting performance and instead led to a decline. This result supports the existing design combining dynamic clustering with temporal attention. It also indicates that further strengthening the association between adjacent partitions may introduce overly strong constraints, limiting the model's ability to adjust grouping promptly when station relationships change rapidly and thereby weakening the adaptivity of patch-by-patch clustering.

LossTemp.U-WindV-Wind
Linfo+RtempL_{info} + R_{temp}6.299 ± 0.0124.068±0.0304.897±0.015
LinfoL_{info}6.291 ± 0.0904.045 ± 0.0284.883 ± 0.013

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.