acceptodds
Withdrawn from ICLR 2027

Transfer-Valid Weather-Station Forecasting: Separating Held-Out Computability from Spatial Communication

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Fixed-station accuracy does not establish transfer to stations excluded from training. We formalize transfer validity for location-known, observation-held-out forecasting as a property of learned prediction under two distinct boundaries. The parameter boundary excludes dependence on identity-indexed trainable state left untrained by the spatial split. Required trainable quantities are instead shared across stations or generated from permitted inputs. The information boundary restricts prediction and routing to data permitted by the split and available at or before the forecast origin. It allows dynamic routes built from the recent history supplied at inference. WeatherScaleNet instantiates these conditions with shared station functions and static-geographic routing. Its completed computational-position graph has diameter at most two when both interaction axes are present, and balanced layouts give spatial attention cost . Five-seed experiments on global weather stations and physical rain gauges evaluate fixed-station accuracy and random-station and contiguous-region holdout. Fixed-station branch controls favor retaining both interaction axes. Coordinate and elevation representations change rank across settings, feature concatenation can outperform scalar aggregation, and partition granularity has limited influence over the tested range. These findings distinguish learned held-out computation, permitted information use, communication structure, and predictive effectiveness without tying transfer validity to one encoder or routing mechanism.

Primary area:
learning on time series and dynamical systems
Keywords:
weather forecasting, spatiotemporal forecasting, spatial transfer, inductive learning, held-out stations, attention networks

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_4wr3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

"Accuracy measured on a fixed set of stations" and "the ability to forecast at stations never seen during training" are two different abilities, and past evaluations conflated the two. Fixed-station evaluation directly demonstrates only temporal generalization to existing stations. Claiming, on that basis, that a model transfers to unseen stations is not supported by sufficient evidence.

Critical strengths
  1. The conceptual separation is genuinely valuable. Separating the parameter boundary (whether held-out prediction relies on untrained identity-indexed state) from the information boundary (whether prediction and routing are restricted to information permitted by the split), and further evaluating accuracy separately from transfer validity, is a clean taxonomic contribution.
  2. The paper raises a valuable question: fixed-station evaluation cannot support claims that a model transfers to unseen stations. It also proposes a new evaluation standard: how to verify that a model has station-transfer capability.
Critical weaknesses
  1. The paper lacks substantial background information and detailed explanation. The Introduction uses a large number of self-invented terms while citing only one reference, which makes it extremely difficult to read.
  2. The paper's novel contribution is held-out station forecasting, yet the corresponding held-out experiments include no baseline comparison (see Table 5). The experiments demonstrate forecasting feasibility, but they do not show that the paper solves the unseen-station forecasting problem, or that it outperforms existing transfer methods.
  3. Ablation experiments. The dual-path ablation does not directly demonstrate that the two paths improve station-transfer capability. This ablation was conducted on fixed-station tasks. It therefore supports: "using local and cross-region communication together improves forecasting at existing stations." It does not support: "the dual paths are the reason for better prediction at unseen stations."
Questions for authors

See weakness

Reviewer_VpmgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work defines transfer validity for forecasting at weather stations held out of training. There are two conditions: no per-station parameters that stay untrained for held-out stations, and no data the split or the forecast time doesn't permit. The method uses shared encoders for station history, coordinates via GeoCLIP embeddings and elevation, plus PatchSTG-style k-d-tree attention within and across station groups. It is evaluated on global wind and temperature data.

Critical strengths

Strengths:

  • The authors carefully separate per-station parameter leakage from information leakage. From my perspective, this is a very useful way to audit held-out-station studies in general.
  • Overall the set of evaluations is as I would expect it: five seeds, paired statistics and block bootstrap, random and region holdouts, real rain gauges with native missingness.
  • I enjoyed reading honest assessments about negative results, e.g. near-zero heavy-rain skill (CSI ≈ 0 at 5 mm).
Critical weaknesses

Weaknesses:

  • held-out-station experiments have no baselines: no kriging, inverse-distance or nearest-neighbour interpolation, persistence, or inductive GNNs. So the core transfer claim is hard to judge for me.
  • overall, I would rank the novelty as rather low. The definition restates standard inductive learning with no leakage, the architecture is PatchSTG plus GeoCLIP, and propositions 1–3 hold by construction.
  • under holdout, using no coordinate encoding works about as well as GeoCLIP, which undercuts the premise that knowing the location helps.
Questions for authors

Questions:

  • How do persistence, kriging, and interpolated GFS/ERA5 perform under the same holdouts?
  • How does WeatherScaleNet do at a parameter budget matched to STELLA (i.e. 100x smaller)?
  • Can you give means and confidence intervals for RQ3?
Reviewer_WE9mOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper separates transfer validity in held-out-station forecasting into two requirements: predictions must not depend on untrained station-specific parameters, and prediction inputs, routes, and auxiliary statistics must respect the spatial split and forecast-time information boundary. WeatherScaleNet instantiates these requirements through shared station encoders, geographical partitioning, and two-axis spatial attention. The experiments separately evaluate fixed-station forecasting, random-station holdout, and contiguous-region holdout.

Critical strengths
  1. The paper usefully distinguishes producing a forecast, supporting it through shared learned computation, respecting information boundaries, and achieving predictive accuracy.
  2. The evaluation covers physical weather stations, rain gauges, random and contiguous spatial holdouts, alternative coordinate representations, and missing-observation settings.
  3. The conclusions are appropriately qualified. The authors acknowledge that GeoCLIP is not consistently best, concatenation can outperform scalar aggregation, and heavy-rain detection and computational cost remain limitations.
Critical weaknesses
  1. Spatial-transfer experiments mainly compare internal variants. Matched external baselines are needed to establish the model’s predictive value at held-out stations.
  2. Communication ablations report qualitative conclusions without numerical effect sizes or confidence intervals.
  3. The two-axis communication controls focus on fixed stations; their benefits under spatial holdout remain insufficiently evaluated.
  4. Routing depends on input station indices, but sensitivity to reindexing is not experimentally assessed.
Questions for authors
  1. Under identical spatial-holdout protocols, how does WeatherScaleNet compare with persistence, shared station-independent models, and other inductive spatial models?
  2. Could you report per-seed communication-ablation results, effect sizes, and confidence intervals, and extend these controls to spatial holdout?
  3. How much do predictions and performance change when the same station set is reindexed? Could a deterministic geographical ordering reduce this sensitivity?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.