acceptodds
Withdrawn from ICLR 2027

SHIFT: INPUT-CONDITIONED STOCHASTIC TEMPORAL REPARAMETERIZATION FOR NEURAL OPERATOR FORECASTING

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Time-series forecasting becomes challenging when observations are irregularly sampled, partially missing, asynchronously recorded, or unevenly informative in physical time. Most temporal neural operators operate on a fixed temporal coordinate system, leaving the allocation of latent temporal resolution deterministic. We introduce SHIFT, a neural-operator forecasting architecture that represents this allocation through an input-conditioned stochastic monotone reparameterization of physical time. Given observed values, masks, timestamps, and time gaps, SHIFT samples positive interval rates whose normalized cumulative sums define a strictly monotone mapping from physical time to a normalized latent-time domain. This stochastic temporal mapping resamples the encoded history onto a uniform latent-time grid, where adaptively fused multi-scale spectral–temporal branches process the resulting representation. Multiple sampled temporal reparameterizations define a finite Monte Carlo predictive mixture, allowing predictive uncertainty to be decomposed into conditional value uncertainty and temporal-geometry-induced uncertainty. The latter measures forecast sensitivity to alternative sampled latent time representations. Across regular, irregular, partially observed, and spatiotemporal benchmarks, SHIFT improves over KRNO in several settings, although the gains remain dataset- and variable-dependent.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-Series Forecasting, Neural Operators, Irregularly Sampled Time Series, Predictive Uncertainty, Stochastic Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_FsDkOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes to apply a learnable time-warping layer prior to feeding a time series into a Neural Operator. To be specific, this layer:

  1. Depends on the input: the input observations and intervals influence the warping;
  2. Is stochastic: the parametrisation is defined by a Gaussian normal distribution;
  3. Preserves chronological order.

To accomodate for that, the authors also introduce a large complex multistep architecture, the "Hierarchical Multi-Scale Temporal Module".

Critical strengths

To the best of my knowledge, this specific combination of parts hasn't been explored by prior work (although I am not an expert in the area). I like the main idea that your paper seems to rest on: that time series has varying informativeness. Indeed: can learnable time warping could improve the quality of the methods; what are the properties of the time series that necessitate such warping, and what cases can serve as counter examples -- these are all very interesting research questions, alas, not tested clearly by this paper. To be clear: this is conjecture on my part, I was not able to find explicit motivation for your work in the paper's text.

Critical weaknesses

As already mentioned in the strengths, I could not find a clean and explicit motivation for your approach in your paper. That is a crucial component for all research papers, arguably more important than empirical results and novelty. Based solely on the main text, I cannot answer with certainty: what question/problem the authors are addressing here.

To be specific, usually in the Introduction, papers describe, what specific weaknesses prior work has, what problems are established for older approaches, and then the sets out to solve, or at least analyze these problems, with the proposed novel approach.

Your paper is built as a detailed technical report for the proposed method. The Introduction section recounts, roughly, what prior work did, and what they do, concluding that prior work didn't do the proposed temporal warping for Neural Operators. Without motivating, why one might be interested in doing such temporal warping, what benefits it may bring, what problems it solves that plagued prior approaches. The other sections also do not explain, what problem is solved, why one might want to solve this problem with this time-warping approach (possibly, what other approaches are to solving this problem, if any).

Consequently, your work establishes no portable findings, that one could then use in another paper, other than that this specific proposed architecture works good on some datasets but not others; without actually specifying, what is required for it to work.

The lack of motivation is not simply a wording issue. Your experiments should be testing that motivation, should be testing your hypotheses and the properties that your method supposedly has. It's beneficial to even lose in some places, given that you show what properties of the benchmark made your method lose -- that's also very valuable.

With the above in mind, the massive 2-4 times wins in the empirical results do not have a clean explanation. Therefore I had to come up with some possible explanations myself, all of them concerning the fairness of the evaluation:

  • Your method has more parameters than the competitors. I cannot tell that based solely on the paper's text, I did not find the competitors' parameter counts. But, I checked against KRNO on Shallow Water -- and your model is ~30x bigger than KRNO. Therefore this isn't really a fair comparison. That is also mirrored by the ablations: Table 7 shows that most of the gains come from the switch from KRNO to your larger multistage hierarchical architecture, not from the main contribution, the temporal warping.
  • The evaluation is not aligned between all methods. The numbers for other methods are copied from their papers, and your code might not be faithfully reproducing the evaluation routine that competitiors use. Indeed, I did find some mismatches, more on them below.
  • The seeds are cherry-picked: 2021, 2024, 2025, 2027 -- these do look suspicious. However, these years do line up with the methods cited, and I don't believe that seed-picking on its own could have produced such a drastic change.

Eval mismatches. These are things to check in your provided code. I may very well be wrong, I did not have the time to reproduce the code or walk over it in detail, the list is from brief targeted reads:

  • You always eval on all channels. KRNO uses last-channel eval for Crypto, for each of the 8 features.
  • Your data preparation, prepare_temporal.py, does not seem to write weights, and therefore the eval may not load weights, scoring RMSE instead of WRMSE on Crypto.
  • On MuJoCo, irregular with missingness, you interpolate the raw inputs prior to passing them into the model. KRNO does not, they handle missing observations via masks (I don't see masks passed into the forecasting model in your code). That could explain your constant score that's seemingly independent of missingness on MuJoCo.
  • You use a 5-step-ahead forecasting routine for Shallow Water in your code. The competitors use a 0.6s recursive rollout, totalling 60 steps (recursively/autoregressively, 5 steps at a time).
Questions for authors

These questions are based on the weaknesses written above, refer there for more detail.

  • How do you motivate your method? I would like to see both prior literature grounding in terms of the problem that your method could solve, some empirical validation for the existance of said problem in real data, for your competitors, and some (ideally also empirically-backed, or theory-backed) explanation as to why your method solves that problem.
  • Are the eval mismatches I highlighted in my review true? Please, check these places, fix where necessary. I would like to see a fair comparison to the competitors, at matched model sizes, horizons, at random seeds, etc. That could go the other way, in which case you will need to be more specific with the choice of benchmarks, to make narrow and verifiable claims, aligned with the motivation.
Reviewer_YPE7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SHIFT, a neural-operator forecasting framework for irregular or partially observed time series. It learns an input-conditioned stochastic monotone reparameterization of physical time, resampling the encoded history onto a uniform latent-time grid for multi-scale spectral–temporal processing. Multiple sampled mappings form a Monte Carlo predictive mixture and support a variance decomposition into conditional and temporal-geometry components. Fixed-coordinate and deterministic-warp controls are used to separate architectural gains from those due to temporal geometry. The reference cost, robustness and sensitivity of the proposed framework are analyzed. It seems to be a solid work.

Critical strengths

(1) The problem is well motivated, since most temporal neural operators operate on a fixed temporal coordinate with deterministic latent resolution allocation, which is indeed limiting for irregular and asynchronous data. (2) The inclusion of fixed-coordinate and deterministic-warp controls is the right methodological choice for attribution and is often missing in comparable work. (3) The paper is well-organized and honest in acknowledging that gains are dataset- and variable-dependent.

Critical weaknesses

(1) According to the evaluation, the core mechanism is not shown to be reliably effective. SHIFT does not consistently improve over its primary baseline KRNO, and in several reported settings it is worse than KRNO and than earlier methods such as T-PatchGNN and Warpformer. (2) The “dataset-dependent” framing is an implicit admission that the central mechanism does not deliver stable gains, so the claims are stronger than the evidence. (3) The attribution results suggest that a substantial part of the gain may come from the broader architecture rather than the stochastic temporal reparameterization. The improvement from KRNO to the fixed-coordinate control is relatively large, while the additional gains from deterministic warp and the full stochastic model appear comparatively modest. (4) The comparison table mixes several method families, and the gap between KRNO and the fixed-coordinate control on some tasks may reflect protocol differences rather than architecture. Systematic comparison against established irregular time series baselines would strengthen the evaluation. (5) Is it possible to compare the proposed framework with existing continuous-time frameworks for irregular time series forecasting?

Questions for authors

see in weakness

Reviewer_CJzjOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

SHIFT is a forecasting model built on KRNO, a neural-operator forecaster, that stretches and compresses the time axis of each input history before forecasting. For every history it samples a random warp that keeps observations in order. Each warp gives some stretches of time more resolution and others less, depending on the input, and the history is resampled onto an evenly spaced grid in this warped time. Multi-scale convolutional and spectral layers then produce a forecast with its own variance. Repeating this over several sampled warps gives an average forecast, and total uncertainty is split into two parts: the average variance of the individual forecasts, and how much the forecasts disagree across warps. Across irregular, regular, partially observed and spatio-temporal benchmarks, SHIFT beats KRNO on most of them. The warp-driven part of the uncertainty usually turns out to be negligible.

Critical strengths

The warp is built simply and soundly: positive interval rates, normalized so warped time always stays in order, and trained end to end. The uncertainty split follows directly from the law of total variance, and the paper is careful about what it means, admitting the warp isn’t uniquely identifiable and that the temporal component depends on the training loss.

The evaluation is broad, covering eight benchmarks across irregular, partially observed, regular and spatio-temporal data, with KRNO rerun directly. Fixed-coordinate and deterministic-warp controls separate the warp’s contribution from the rest of the architecture, and the probabilistic forecasts are scored with proper metrics.

The paper is also candid. It reports its failures on MIMIC-III and the Shallow Water velocities, gives full per-dataset settings and inference costs, and uses small models that run on a single older GPU.

Critical weaknesses

The paper does not support its own claims. The warp is an input-dependent resampling of the history, and the paper admits it can’t be identified apart from the rest of the network. The stochastic version beating the deterministic one is more simply explained by the random noise acting as a regularizer during training. The Shallow Water identity-warp test only shows that a model trained with warps breaks when they are removed.

The uncertainty split is close to nothing. Temporal uncertainty is about 10⁻⁷ against 10⁻² on USHCN, effectively zero on MIMIC-III, and zero by construction on M4 Quarterly. It also rises nearly thirteenfold on Crypto when the training loss changes. The paper never checks whether it tracks forecast error, and calibration is shown on only two datasets with no baseline.

The accuracy gains mostly come from elsewhere. Without any learned warp, the model already cuts Crypto error from 6.95 to 3.00, against 2.98 for full SHIFT, and is worse than KRNO on USHCN. Each dataset gets its own tuned losses, regularization and sample counts, and the paper describes no comparable tuning for the KRNO reruns. Some margins are surprisingly large to the point of implausibility for a small model on established benchmarks, such as 57% on Crypto and 77% on Player Trajectory, and go unexplained. SHIFT also clearly loses to KRNO on MIMIC-III.

Questions for authors

Furnish the actual proof that the warp does what the paper claims. The authors should compare it against three alternatives: random warps that ignore the input, the same noise injected as input noise or dropout, and a learned warp that doesn’t depend on the input. If these match SHIFT, the benefit is regularization, not input-dependent geometry. The identity-warp test should be replaced by evaluating the model with warps sampled for other inputs, and a synthetic task with known informative regions would show whether the warp expands the parts of the history that matter.

Show that the uncertainty split contains real information or drop it as a claim. Every dataset should be trained with the mixture likelihood already tested on Crypto, with more than one sample on M4 Quarterly. The authors should check whether temporal uncertainty tracks forecast error and rises under heavier missingness. Calibration should cover every dataset and be compared against KRNO with a variance head, deep ensembles, and the probabilistic neural operator and ProFITi models the paper already cites.

Show where huge accuracy gains come from. KRNO should get the same per-dataset losses and tuning budget, or SHIFT should use one shared objective, and the fixed-coordinate model belongs in the main tables. For the 57% and 77% gains on Crypto and Player Trajectory, the authors should confirm that the splits and metric scaling match KNF and really zero in on the source of gains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.