acceptodds
Under review as a conference paper at ICLR 2027

Level-Shift Failures in Spatiotemporal Kriging with Target-Node History: Diagnosis and Repair

Abstract

We study level-shift failures in spatiotemporal kriging when target nodes' current values are masked but their histories remain available. KRIN subtracts each node's recent mean and restores it after prediction. Under a common reconstruction protocol, it reduces MAE in 37 of 40 configurations using released implementations. However, none of the centred released implementations outperforms seasonal persistence with current common-mode correction (+CM) on Chicago or Subway under Shock. Controlled experiments examine how recovery varies with the location and timing of available level information. Holding targets and observation counts fixed, moving observations outside a shifted region increases SPIN-s MAE from 194.6 to 598.4 on Subway, while SPIN-s + KRIN changes from 121.2 to 123.9. Immediately after a uniform shift, +CM achieves lower error than the learned configurations, while shift alarms identify failures after centring only imprecisely. These findings distinguish broad error reduction from conditional gains over a strong baseline and highlight sensitivity to observation location and transition timing in the evaluated settings.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Spatiotemporal kriging, level shift, distribution shift, target-node history, normalization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8TnxOpenReview ↗
Rating
2(+1)
Confidence
3(+1)
Informativeness
2(+1)
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies performance degradation in spatiotemporal kriging reconstruction under level shifts. It analyzes the link between shifts and model failure, especially when neural models underperform simple historical baselines, and proposes KRIN, which improves robustness by removing and restoring each node’s recent mean. Experiments across models and datasets, including matched comparisons of observation-node locations, show reduced prediction errors for most models.

Critical strengths
  1. The evaluation is systematic and conclusions are cautious. It compares models under normal and level-shift conditions, distinguishes simplified architectures from public implementations, examines observation-node locations through matched comparisons, and reports cases where KRIN fails to beat strong baselines. This strengthens the findings’ credibility.
Critical weaknesses
  1. Limited novelty. KRIN removes each node’s historical-window mean and restores it after prediction, closely resembling RevIN [1], which removes and restores instance statistics, and NLinear [2], which subtracts and restores the last observed value.
  2. Gains over existing centering methods appear limited. Table 23 suggests that most improvements stem from generic level centering rather than KRIN’s specific recent-window mean design.

[1] Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., & Choo, J. Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift. ICLR 2022.

[2] Zeng, A., Chen, M., Zhang, L., & Xu, Q. Are Transformers Effective for Time Series Forecasting? AAAI 2023, 37(9), 11121–11128.

Questions for authors
  1. Table 4 shows that KRIN nearly eliminates the additional baseline-normalized outside-observation penalty caused by local shifts. However, this may follow from its equivariance and synthetic shift construction. How much of this result is an empirical finding beyond the mathematical structure?
  2. Why does KRIN substantially outperform SNaive-7+CM on some simplified architectures, while the corresponding public implementations still underperform this baseline?
3 comments

e thank the reviewer for these questions.

W1. Contribution beyond RevIN and NLinear.

KRIN removes and restores a historical mean without window scaling or a learned affine; it adds no new operator. Our contribution concerns failures against admissible baselines and level sources. A statistic using current values must come from observed nodes. Table 66 compares sources on the same backbones without added parameters. In Chicago's settled local-shift test, SPIN-s with an observed-node level matches KRIN inside the shifted region (29.7 vs 30.1 MAE), but outside has 4.9 times KRIN's MAE (149.8 vs 30.5), above the uncentred model (102.8). The outside ratio is also 4.9 in Subway (627.5 vs 127.2).

W2. Does the gain come mainly from centring?

Yes (§5.1). The former “full RevIN” row used unbiased standard deviation plus 1e-5 without a learned affine. It is relabelled and RevIN as published is added (Tables 23, 62–63). Across SPIN-s and IGNNK-R, with 3 training seeds × 10 mask seeds per cell, paired intervals favour the mean in 6/16 comparisons and RevIN in 5; 4 include one. In IGNNK-R Airports 2020–21, 5/30 evaluations from two training seeds meet the extreme criterion: non-finite MAE or MAE above three times the KRIN median. All five are finite; excluding them gives RevIN MAE 8.86 versus 8.58 for the mean. Maximum shortfalls (mean / RevIN) are 11.1% / 11.4% on SPIN-s and 7.0% / 10.2% on IGNNK-R excluding that cell; last-value removal gives 15.2% on SPIN-s and 17.6% on IGNNK-R. These results establish neither equivalence nor overall superiority; the mean is a parameter-free default.

Q1. What does Table 4 show beyond mathematical cancellation?

The near-zero centred change is expected: the shift starts 14 days before evaluation, so the full history shares it. Exact cancellation holds for additive offsets in preprocessed space; raw-count multiplication differs slightly after log1p. We treat this as an implementation check (§7.2). We also measure:

  1. Uncentred penalty. Excess outside-observation penalty D is 0.41–1.38 (Table 4).
  2. Abrupt onset (Table 65). We used 19 onsets, each scored for 28 days. Errors are pooled over the 19 onsets within each interval before the ratios are formed, and the intervals resample training seeds and placements; the onsets are not 19 independent shocks.

For onset, we fix the normalisation reference at α times the no-shift SNaive-7 MAE rather than the contemporaneous SNaive-7 error used in Table 4. During transition, the weekly reference still includes pre-shift observations. The fixed reference prevents that baseline's transition error from rescaling the placement contrast. R is the reduction in excess penalty, not absolute MAE recovery (means, 95% crossed-bootstrap intervals):

Backbone, cityDays 1–7Days 8–14Days 15–28
IGNNK-R, Chicago−1.127 [−1.81, −0.55]0.618 [0.41, 0.92]0.432 [0.37, 0.51]
IGNNK-R, Subway−1.447 [−1.95, −0.94]0.230 [0.07, 0.36]0.422 [0.24, 0.56]
SPIN-s, Chicago−1.366 [−3.06, −0.08]−0.001 [−0.44, 0.40]1.015 [0.30, 1.55]
SPIN-s, Subway−0.280 [−2.17, 1.59]−0.066 [−0.71, 0.67]1.387 [1.14, 1.71]

Mean R is negative in all four pairs in days 1–7, positive for IGNNK-R and near zero for SPIN-s in days 8–14, and positive with intervals above zero in all four in days 15–28. Original checkpoints were not retained. Retraining with the same code, settings and seeds reproduces Table 4 for IGNNK-R to numerical precision, but not SPIN-s exactly; both original SPIN-s estimates fall within the rerun intervals. Negative R does not imply higher IGNNK-R MAE: Chicago error falls by 37% inside and 12% outside, widening the gap. SPIN-s outside instead has higher first-week centred MAE (439.2 vs 370.4; Table 66).

  1. Level source. The observed-node level gives excess D about 2.0 in three pairs and 0.55 [−0.73, 1.54] for IGNNK-R on Subway. This concerns the controlled local shift.

Q2. Why does SPIN-s outperform SPIN-R against +CM?

The cause remains unresolved. Only SPIN-s and SATCN-s (2/6 centred simplified models) beat +CM on Chicago/Subway Shock at r = 0.5, versus 6/6 on Bikeshare and 5/6 on Airports (Table 16).

  • Even the best tested SPIN-R + KRIN seed has higher MAE than +CM: 225.5 vs 86.7 in Chicago and 577.3 vs 543.4 in Subway.
  • Restricted SPIN-s stays below +CM in all 30 evaluations per cell. Restricted/global Shock ratios are 1.006 [0.982, 1.042] and 1.064 [1.031, 1.098]; ratios to +CM are 0.810 and 0.850 (Table 70).

Both share inputs, preprocessing, KRIN, masks, checkpoint selection and final-target masked MSE. SPIN-s uses a GRU and attention across observed nodes (hidden 64, batch 32); SPIN-R uses four sparse spatiotemporal attention layers and learned node embeddings (hidden 32, batch 4). Both omit original SPIN's layer-wise and whole-window losses. These controls do not isolate an architectural cause (Tables 25, 69; §7.5).

I thank the authors for their patient and detailed responses. I learned several new things from them, and they have addressed my questions. However, the paper still has some shortcomings, and I remain concerned about its readability, as I still find parts of it difficult to follow. Nevertheless, in light of the authors’ clarifications, I have decided to raise my score.

Thank you for your careful reading and thoughtful feedback, and for raising your score in light of our clarifications. We are glad that our responses addressed your questions.

We take your remaining concern about readability seriously and would like to use the rest of the discussion period to act on it. If you are able to spare a moment, we would be very grateful if you could point us to one or two sections or passages that you still found difficult to follow. Even a brief pointer would help us direct the revision to where it matters most for readers.

We fully understand if time does not permit, and we will continue improving the paper's organisation and clarity in any case. Thank you again for your generous engagement with our work.

Reviewer_LUMXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates level-shift failures in spatiotemporal kriging when target-node histories remain available but current-day observations are masked. The authors propose KRIN, a parameter-free normalization strategy that removes each node's recent historical mean and restores it after prediction. Experiments across multiple spatiotemporal models and datasets show that KRIN reduces reconstruction errors in 37 of 40 configurations using adapted released implementations. The paper further examines how observation locations, historical data availability, and level-shift transitions affect reconstruction performance, highlighting both the effectiveness and limitations of normalization under distribution shifts.

Critical strengths
  1. Valuable empirical analysis of level-shift failures. The paper systematically investigates how observation locations and the availability of target-node history affect the recovery of spatiotemporal models under level shifts. These analyses provide useful insights into the conditions influencing model robustness and reconstruction performance.

  2. Effective and lightweight mitigation strategy. The proposed KRIN introduces no additional trainable parameters and demonstrates substantial performance improvements across multiple spatiotemporal models. In particular, it reduces MAE in 37 out of 40 configurations using adapted released implementations, demonstrating the practical effectiveness of the normalization strategy.

Critical weaknesses
  1. Limited methodological novelty and insufficiently differentiated contributions. KRIN largely adapts the well-established mean-removal and restoration strategy of RevIN and NLinear to spatiotemporal kriging, without introducing a substantially new architectural or learning mechanism. Moreover, the comparisons in Appendix B.2.2 show that existing normalization approaches achieve comparable or even better performance in certain settings. Although the authors position the paper as an empirical study, the findings do not convincingly establish sufficiently novel and generalizable insights beyond the known benefits of normalization under distribution shifts. Consequently, the overall contribution appears largely incremental, which significantly weakens the paper's case for acceptance at ICLR.
  2. Insufficient motivation and questionable representativeness of the task setting. Although spatiotemporal kriging is a well-established research problem, this paper focuses on a restricted setting where only the target-day observations are masked while historical observations remain available. The Introduction does not sufficiently explain the practical significance or prevalence of this specific setting. Moreover, the experimental masking ratios (25%–75%) are substantially higher than the naturally occurring missing rates reported in the datasets (0%–3.95%). This discrepancy weakens the evidence supporting the method's effectiveness in real-world applications.
Questions for authors

Practical significance and representativeness of the task setting. Could the authors provide concrete real-world application scenarios where only the target-day observations are missing while historical observations remain available, and clarify how common this setting is in practice? Furthermore, given the substantial gap between the simulated masking ratios (25%–75%) and the naturally occurring missing rates (0%–3.95%), how can the experimental results demonstrate the method's effectiveness in realistic scenarios?

3 comments

We thank the reviewer for the questions on contribution, task setting and masking ratios.

W1. Limited methodological novelty and insufficiently differentiated contributions.

KRIN follows the level-removal and restoration principle of RevIN and NLinear and adds no new architectural or learning mechanism; the gain is largely shared across centring statistics (§5.1). Against RevIN as published (Tables 23, 62, 63), intervals favour the window mean in 6 comparisons and RevIN in 5, 4 include one, and in one cell 5 of 30 RevIN evaluations exceed three times the KRIN median error (all five are finite; Table 62); we claim no overall equality. §1–§2 now state the empirical contributions:

  1. Failure against admissible training-free baselines. At r = 0.5 on the three shifted datasets, all 15 uncentred released-model configurations have higher mean Shock error than SNaive-7, and none of 450 evaluations has lower error than +CM (Table 64). GRIN and SPIN omit temporal persistence from their baselines (Appendix B.2.6).
  2. The source of the level. With SPIN-s and a settled local shift, the observed-node level matches KRIN inside the shifted region but has 4.9 times KRIN's error outside, above the uncentred model; in the first week it is better when observations share the shift (Table 66).
  3. The limits of centring. Lower error than an uncentred model does not imply lower error than +CM: no centred released model beats +CM on Chicago or Subway under Shock at r = 0.5 with the main-protocol checkpoints, and the low-masking results below depend on training masks.

On generalisability, §5.3 and Table 67 give level-only mismatches for ideal steps and linear drift; Table 68 finds positive Spearman correlations between nodewise KRIN / +CM remainder and MAE ratios (7 of 8 intervals exclude zero), an association that does not determine total error. All large real shifts come from one pandemic event (§7.5, §8).

W2 and Question (first part). Real-world setting and prevalence.

Scenario. One station's daily gate counts or one airport's daily movements are temporarily missing while its earlier records and other locations' current records exist, and an operator needs that day's value. Like GRIN/SPIN imputation, the task uses other observations of the same sensor, but it reconstructs only the last day from earlier history (§2).

Prevalence in our data (§3, Tables 28, 60):

AirportsSubway
Node-level reporting gaps3,326536, at 188 of 437 stations
Lasting exactly one day74%86%
Lasting at most seven days97%99.6%
Missing nodes on a day with gaps (median / 90th percentile)1 / 41 / 4

Subway's 3,496 episodes on 56 network-wide outage days are counted separately; mixing them gave the submission's 11%. Bikeshare and Chicago have no internal gaps, so this evidence covers two of four datasets and is not a prevalence estimate for other networks. A value hidden on day t can enter the history on day t+1; never-observed nodes and prolonged or network-wide outages are outside the main task (§3, §7.5).

W2 and Question (second part). 25–75% masking versus 0–3.95% natural missingness.

The units differ: r is the fraction of nodes hidden on the evaluated day, whereas Table 2 counts node–day cells. Excluding Subway's outage days lowers its rate from 3.95% to 0.12%; on a gap day the median is one missing node (0.45% of Airports, 0.23% of Subway). We call r = 25–75% a stress range (§6) and added r = 1%, 5% and 10% (§7.1, Table 61): released IGNNK-R with and without KRIN on Chicago, Subway and Airports, 3 training seeds × 100 mask seeds (at r = 1%, 1, 4 and 2 nodes are hidden). Shock MAE ratios at r = 1% / 5% / 10%:

DatasetKRIN / uncentred, main-protocol checkpointsKRIN / +CM, main-protocol checkpointsKRIN / +CM, trained with r ∈ {0.01, 0.05, 0.10}
Chicago0.57 / 0.57 / 0.581.40 / 1.24 / 1.230.97 / 0.89* / 0.91*
Subway0.55 / 0.56 / 0.561.44 / 1.40 / 1.351.06 / 1.03 / 1.03
Airports0.80 / 0.81 / 0.790.93* / 0.91* / 0.91*0.91* / 0.90* / 0.90*

An asterisk denotes a 95% interval entirely below one. Intervals are from a crossed bootstrap over training seeds and mask seeds.

  1. Centring reduces error at lower ratios: the KRIN / uncentred upper bound is below one in 33 of 36 cells (both conditions and training schemes).
  2. The comparison with +CM depends on training masks. Main-protocol checkpoints (trained at r ∈ {0.25, 0.5, 0.75}) remain worse than +CM on Chicago and Subway. Low-ratio training gives intervals below one in 14 of 18 Shock/Normal comparisons (Subway Shock near parity, 1.03–1.06), but those models are much worse than +CM at r = 0.5 (Chicago Shock: 153 vs 87).

We therefore recommend checking the match between training and deployment masks (§8); one released backbone and random masks do not cover every operational pattern.

Thanks for authors' rebuttals. However, the presentation of this paper is not clear enough. I still believe my initial rating is fair. Therefore, I would like to keep it.

Thank you for revisiting our work and sharing your assessment. We sincerely appreciate the time and thought you have devoted to this discussion. Your questions have helped us sharpen both the contribution and the practical scope of the paper.

We take your continued concern about clarity seriously and remain committed to making the paper easier to follow.

Alongside our written responses, we have also updated the manuscript PDF. To make the changes easier to locate, we briefly summarize the revisions most relevant to your concerns:

  • Contribution and positioning (§§1–2, §5.1): We explicitly distinguish the established centring principle from our empirical contribution: examining how the source of level information, observation placement, and transition timing affect reconstruction. We do not claim a fundamentally new normalization mechanism or consistent superiority over existing centring approaches.
  • Practical setting and scope (§3, §7.5): We clarify the scenario of temporarily missing current measurements with historical records available, add evidence on the duration and frequency of reporting gaps, and explicitly identify settings outside the study’s scope.
  • Realistic missingness versus stress testing (§6, §7.1): We distinguish the higher masking ratios used for stress testing from naturally occurring missingness, and add evaluations at 1%, 5%, and 10% masking. These results also make clear that performance depends on the match between training and evaluation masks.
  • Interpretation and limitations (§§7–8): We separate improvements over uncentred models from gains over the strong training-free baseline, and explicitly discuss transition lag and the limited diversity of large real-world shocks.

These revisions aim to make both the findings and their boundaries more transparent.

If there is a particular passage or argument you would like us to clarify during the remaining discussion, we would welcome the opportunity to address it directly. Thank you again for your careful consideration and constructive engagement.

Reviewer_kTYAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates level-shift failures in spatiotemporal reconstruction with available target-node histories and introduces KRIN, a historical-mean centering wrapper. Its main contribution is an empirical analysis of how level handling and observation placement affect performance, supported by strong training-free baselines. The findings are useful and carefully qualified, although methodological novelty and generalizability remain limited.

Critical strengths
  1. Strong baseline comparisons clearly distinguish improvements over uncentered models from gains over seasonal persistence with common-mode correction (+CM).
  2. Matched no-shift and local-shift experiments help isolate the additional effect of observation placement under shifts.
  3. KRIN is simple, adds no trainable parameters, and reduces MAE in 37 of 40 configurations using adapted released implementations.
Critical weaknesses
  1. Mean removal and restoration closely follow existing normalization approaches.
  2. The main protocol assumes available target histories and one-day gaps, while large real-world shifts come from the same pandemic event.
  3. No centered released model outperforms +CM on Chicago or Subway, and KRIN can struggle immediately after abrupt transitions.
Questions for authors

See above

3 comments

We thank the reviewer for identifying these limits. The revision makes them explicit and adds analyses of when each level source helps or fails.

W1. Mean removal and restoration closely follow existing normalisation approaches.

We agree; the abstract relates KRIN to RevIN and NLinear. The contributions are empirical: admissible training-free baselines, evidence for a level-handling failure mechanism, the choice of level source, and transition errors and same-day alarms (§1; general response).

W2. The main protocol assumes available target histories and one-day gaps, and large real-world shifts come from the same pandemic event.

  • Airports has 3,326 node-level gaps (74% one day); Subway has 536 after separating network-wide outages (86% one day). The median number of missing nodes on a gap day is one. Bikeshare and Chicago have no internal gaps, so this evidence covers two of four datasets (§3, Table 60).
  • With contiguous blocks removing 22.1% of history entries (median length five days), centring improves all 24 configurations (Appendix C.1.1). Removing all target history falls back to a zero fill in standardised space (the training mean): Chicago KRIN MAE rises from 126.1 with one retained slot to 1,182.8 with none (§7.4, Table 52).
  • Never-observed nodes and prolonged or network-wide outages are outside the main task (§3, §7.5).
  • The three shifted networks share one real event; the abstract, limitations and conclusion state that all large real shifts come from one pandemic event. Controlled local/global and settled/onset shifts vary conditions within the studied data; NO2 and METR-LA are low-shift comparisons (Appendices C.6 and D), not independent large-shock replications.

W3. No centred released model outperforms +CM on Chicago or Subway, and KRIN can struggle immediately after abrupt transitions.

+CM on Chicago and Subway. In the main protocol under Shock at r = 0.5, no centred released model has lower error than +CM in any of 300 evaluations (§7.1, Table 64); the abstract retains this result. §5.3 and Appendix G compare level components in log1p units. For an ideal persistent additive step with τ = 0 on the onset day, KRIN leaves δ_i(K−τ)/K for 0 ≤ τ < K and zero afterwards; +CM leaves the target step minus the observed-node median step for τ < 7; the observed-node mean leaves the target step minus the observed-node mean step. For established linear drift, KRIN leaves s_i(K+1)/2 and +CM leaves seven times the target slope's departure from the observed-node median slope (Table 67).

These are level mismatches, not total MAE. Table 68 estimates a smaller +CM remainder at 140 of 143 Chicago and 425 of 435 Subway nodes in 2020, consistent with +CM's strength under common movement. Measured errors (Table 66):

ConditionIdeal additive level termMeasured MAE, Chicago
Days 1–7, observations inside a shared step+CM and the observed-node mean cancel the step; the history mean lagsIGNNK-R: +CM 39.0, observed-node level 53.0, KRIN 223.0
Days 1–7, observations outside the shifted region+CM keeps the full step (pre-onset weekly reference, no common change observed); the history mean has partly absorbed it; the observed-node mean misses itIGNNK-R: KRIN 493.3, observed-node level 488.8, +CM 1,206.1
Settled step, observations outsideHistory and weekly references have caught up; the observed-node mean misses the stepSPIN-s: KRIN 30.5, +CM 39.4, observed-node level 149.8

The per-node test on simplified SPIN-s and IGNNK-s finds positive correlations between KRIN / +CM remainder and MAE ratios (intervals excluding zero in 7 of 8 pairs): an association, not proof of the released models' ordering. Under Normal, centred IGNNK-R and GRIN-R on Chicago and GRIN-R on Subway have lower error than +CM in 30 of 30 evaluations (§7.1), so the shortfall concerns the tested Shock condition. §8 recommends validating the level choice and mask distribution against +CM.

Immediately after abrupt transitions. In the original global-shift test, first-week MAE is 12.2 and 70.4 for +CM versus 27.1 and 234.5 for SPIN-s + KRIN in Chicago and Subway (Tables 40–41). In the local-onset tests (Tables 65–66), errors are pooled over days and onsets before ratios are formed, using the fixed α × no-shift SNaive-7 denominator. Mean KRIN reduction R in the excess outside-observation penalty is negative in days 1–7 for all four backbone–city pairs; in days 8–14 it is positive for IGNNK-R (0.618 [0.41, 0.92] Chicago, 0.230 [0.07, 0.36] Subway) and near zero for SPIN-s; in days 15–28 all four are positive. Centred SPIN-s outside the region has higher first-week MAE than its uncentred version (439.2 vs 370.4, Chicago). The revision separates this adaptation lag from absolute error and the placement penalty; a vanishing ideal mismatch after K days guarantees neither accuracy nor superiority over either baseline.

Thanks for the authors' reply, and some of my concerns have been addressed. I will keep my score.

We thank the reviewer for their thoughtful feedback and for taking the time to consider our responses. We are glad that our clarifications have addressed some of your concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.