acceptodds
Under review as a conference paper at ICLR 2027

The Exact Risk of Instance Normalization

Abstract

Instance normalization (IN), which removes each input window’s statistics before forecasting and restores them afterward, is standard in time-series forecasting, yet there is little theory saying when it helps or fails. We analyze linear forecasters, starting from a characterization of mean-only IN as least squares constrained to weights summing to one, and under ridge training as ridge centered at the uniform vector on that plane. This yields closed forms for IN’s cost and benefit under unseen level shift, with a boundary: IN wins when the squared shift exceeds the variance of the best linear estimate of window level. Three consequences follow. First, the choice need not be binary. Interpolating the two forecasters admits an exact risk decomposition around any backbone, linear or not: the optimum is interior precisely when price and benefit are both positive, its excess risk is the parallel combination of the two endpoints’, and in the linear case it is the shift-to-boundary ratio , computable before any label is seen. Second, division by the window standard deviation cancels identically in linear prediction, so it cannot amplify test error; it acts through training, reweighting examples by , and the scale of that reweighted loss, not weight concentration, separates two catastrophic failures from six benign cases on eight benchmarks, from training data alone. Third, finite-sample estimation noise creates shift fragility even when the population model is shift-insensitive. Three preregistered experiments on PatchTST and iTransformer, outcome texts frozen before any run, mark the mechanism’s reach and its limit: the deep statistic ranks normalization gains at all four horizons (–) but is shown not to predict their sign, since it estimates the benefit and carries no price term; both regimes realize on transformers; and the interior optimum is found in 8 of 8 deep cells, where, estimated without test labels, it cuts regret against the oracle by 17× relative to any binary rule.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series forecasting, instance normalization, RevIN, distribution shift, linear models, non-stationarity

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2jkfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper develops an exact risk theory for instance normalization (RevIN) in linear time-series forecasters and evaluates the resulting diagnostics on eight benchmarks and in preregistered experiments on deep transformers.

Critical strengths
  1. The paper elevates the study of instance normalization from representational algebra to an exact risk theory, yielding closed-form expressions for the stationary price versus level-shift benefit.
  2. The author has conducted extensive verification work, and the authors evaluate transfer to PatchTST and iTransformer using frozen, multi-tier preregistrations that report every failed gate and falsified prediction verbatim.
Critical weaknesses
  1. A large number of unexplained parameters appear in the paper, including those in the abstract. This makes the paper very difficult to understand.
  2. No relevant prediction theories have been mentioned in recent work. In addition, a large number of py files are presented in the appendix, yet no relevant results are provided. This is the theoretical basis on which I doubt this work.
Questions for authors
  1. Can the method proposed in this paper explain other normalization methods (except RevIN)?
  2. Why is there no comparison with other normalization methods?
  3. Can you explain in simple terms why instance normalization improves performance?
2 comments

We thank the reviewer for the constructive feedback and for recognizing the value of the exact risk analysis and the preregistered experiments.

We have revised the manuscript to address the concerns regarding presentation, related theory, and comparisons with other normalization methods. We respond to the three questions below.

Q1. Can the theory explain normalization methods other than RevIN?

Yes, for a defined class of normalization procedures, although the results do not extend to every learned normalizer.

We have added Table 2 (p. 10) and Appendix B.6 (p. 18) to clarify this scope.

First, any method that subtracts a fixed weighted average of the input window and restores it after prediction, where the weights sum to one, produces the same class of linear forecasters as mean normalization. This includes last-value normalization, as used in NLinear, and fixed exponentially weighted means.

Under unregularized least squares, these methods yield the same fitted forecaster and inherit the same exact risk results. Under ridge regularization, their induced penalties differ, so they need not produce the same fitted estimator.

Second, first differencing spans the same linear hypothesis class, as established in Proposition 4.

Third, Theorem 4 extends the population analysis to known nuisance directions, including level, trend, and fixed seasonal components.

Finally, for linear forecasters, division by a positive window-scale statistic cancels at prediction time but changes the weighting of the training loss (Section 5).

Methods that introduce learned statistics or additional interactions, including Dish-TS, SAN, and NST, are not covered by these results. Table 2 distinguishes the applicable cases from those requiring separate analysis.

Q2. Why are there no comparisons with other normalization methods?

We agree that the original manuscript did not make the existing comparisons sufficiently visible.

The revised Section 9 (pp. 9–10) now brings together the results for four estimators evaluated under the same linear forecasting protocol: no normalization, mean normalization, full instance normalization, and differencing.

Across 32 dataset–horizon settings, mean normalization achieves the lowest reported test MSE in 13 settings, the unnormalized model in 12, differencing in 5, and full instance normalization in 3. One tie is counted for both mean normalization and the unnormalized model.

Mean normalization and differencing differ by less than 0.01 MSE in 27 settings, consistent with their shared hypothesis class but different regularized estimators.

We also report an explicitly post hoc comparison involving full RevIN on the deep backbones in Appendix S.

We acknowledge that we have not conducted experimental comparisons with learned normalizers such as SAN or Dish-TS. The new Table 2 provides an analytical comparison, but this does not replace a direct empirical benchmark.

Q3. Why does instance normalization improve forecasting performance?

The intuition is that instance normalization makes the forecast respond to a common change in the level of the input and target.

Suppose a model was trained on a series centered around zero, but the series subsequently shifts to a higher level. An unnormalized model may continue forecasting toward the old mean, producing systematic error.

Mean normalization removes the current window level before prediction and restores it afterward. The resulting forecast responds to the shift exactly, avoiding this source of error.

However, this response has a cost. If fluctuations in the window mean are temporary rather than persistent, normalization forces the forecast to follow them.

Our main result quantifies this trade-off. For unregularized population least squares, normalization is preferable precisely when the squared deployment shift exceeds the variance of the best linear unbiased estimate of the window level.

Section 1.1 (pp. 1–2) now develops this explanation through a numerical forecasting example.

Additional clarification of novelty and related work.

We have expanded the discussion of recent forecasting theory, including Toner and Darlow (2024), Huang et al. (2026), and Wang et al. (2026). We also cite Herbreteau et al. (2023) and Saied and Fleuret (2026) for established normalization-equivariance properties.

We do not claim novelty for the sum-to-one identity or the general equivariance of the normalization wrapper. Our contribution is the exact risk analysis and its consequences for forecasting.

Finally, Appendix U now summarizes verification results alongside their corresponding claims and scripts, rather than presenting raw console output without a clear connection to the theoretical statements.

We hope these revisions address the main concerns and clarify both the contribution and its limitations.

Unfortunately, I am not fully convinced. Admittedly, the paper is valuable, yet there is a critical discrepancy between the quantity required by the theory and the quantity actually estimated in the experiments.

Reviewer_wYHMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper is unintelligible as written. To my understanding, the authors aim to analyze the theoretical properties of Instance Normalization (IN) which is a common practice in time series forecasting where the mean of the input window is subtracted before doing the prediction. Allegedly, the paper addresses the exact cost and benefit of doing so, and identify settings where the benefit is worth the cost. That said, I could not parse most of the paper given how poorly written and structured it is. The writing is stiff and unnatural throughout with constants and symbols being used before being defined and explained, (e.g. aa and bb in the introduction. In fact, I don't think bb was defined anywhere in the paper). Propositions, Theorems, and Corollaries are thrown at the reader without any explanation as to their importance, significance, or any intuition that might help understand them.

Critical strengths

N/A. See my summary above

Critical weaknesses

1- The writing and structure of the paper is unintelligible and borderline unreadable.

2- The contributions are not clearly presented and nearly impossible to understand. Furthermore, the introduction presents the contributions in a way that is difficult to parse without reading the rest of the paper.

3- Theoretical results are presented suddenly and without context or intuition building to help the reader grasp their significance.

Questions for authors

In my opinion, the paper as it stands is unfit for submission. I would ask the authors to completely rewrite the paper, focusing on understandability and clarity. The contributions should be stated in a readable way, and with the proper context so the reader can understand their significance.

2 comments

We thank the reviewer for the candid feedback. We recognize that the original manuscript did not adequately prepare the reader for its mathematical results. In particular, the introduction introduced technical quantities before explaining their roles, and several theorems were presented without sufficient motivation.

We have substantially rewritten and reorganized the manuscript to address these concerns. The core theoretical results and experimental findings remain unchanged. The principal revisions are as follows.

  1. A more accessible introduction (Section 1.1, pp. 1–2).

We now introduce the central question through a concrete AR(1) forecasting example before presenting any general theorem. The example shows why instance normalization can help when the level of a time series changes, why it can hurt when the series remains stationary, and how these effects lead to an exact risk threshold.

For this example, normalization increases the stationary mean squared error from 1.7301 to 1.8974, but becomes preferable when the deployment shift exceeds approximately 0.62. We derive this threshold and explain its statistical interpretation before introducing the general formulation.

  1. Clearer definitions and contributions (Sections 1.2–2, p. 3).

We have removed unexplained mathematical symbols from the abstract and introductory exposition. The setup now defines the relevant quantities before they appear in formal results, and Appendix A provides a notation reference.

  1. Motivation and intuition for the main theorems (Sections 3–6, pp. 4–7).

We have reorganized the theoretical sections so that each major result is introduced by the question it answers.

In particular, Section 3 now explains the central risk decomposition before presenting its complete proof in Appendix B. The normalized optimum differs from the unrestricted optimum by a correction based on the best linear unbiased estimate of the window level. The variance of this correction determines the stationary cost of normalization, while the same coefficient governs its benefit under a shift.

This gives an intuitive interpretation of why the risk threshold takes its particular form.

  1. Improved presentation of evidence and limitations.

We have clarified the figures, tables, and empirical conclusions, including the distinction between ranking the benefit of normalization and predicting whether normalization improves forecast accuracy.

We hope these changes make the theoretical contribution easier to assess. We would welcome any further questions about the assumptions or derivations of a particular result.

I thank the authors for their rebuttal. The paper's readability has improved somewhat to the extent that I now understand the main conceit of the paper more clearly. Namely, that for a time series that has a constant level shift compared to the original training data, IN has a benefit that is increasing in the size of the level shift, and has a cost that depends on the variance of the mean level that can be estimated from any give window. Thus whenever the benefit exceeds the cost, IN should lead to smaller loss compared to an unnormalized approach.

That being said I still have the following major issues with the paper:

1- The readability and presentation are still an issue. Especially, the experiments section where it is still quite difficult to parse what the experiments are actually saying and why were the datasets split between the different 'tiers'.

2- To my understanding, from a theoretical perspective at least, the central result (which I outlined above) seems quite simple. As such given that the experiments are also poorly presented, I remain unconvinced of the significance and novelty of the main result.

3- The revised submission is significantly different to the original submission to an extent that is much larger than usual changes from submission to rebuttal. While according to the authors, the main results have not been changed, at least the presentation and structure has been completely overhauled.

Due to the above reasons I have decided to maintain my score.

Reviewer_ufLvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when instance normalization helps time-series forecasting. For linear models, mean normalization makes the effective weights sum to one. The paper derives the cost of this constraint and its benefit when the data level shifts. It also studies finite training samples and the effect of dividing by the window standard deviation. SoftIN mixes the raw and mean-normalized predictions. Tests include eight linear benchmarks and several stages of deep-model experiments. The theory is useful. The experiments also show limits: the diagnostic does not always predict whether normalization helps, and the practical SoftIN test is small.

Critical strengths
  1. The theory explains a common forecasting choice. It connects the cost of normalization, the size of a level shift, and the amount of training data (Sections 3–5). It also shows that weight estimation can create shift sensitivity even when the population model has none. This is my main reason to support acceptance.

  2. The authors report where the deep-model diagnostic does not transfer. These include iTransformer on Traffic, the failed Tier-2 predictions, and the unsupported deep NTE hypothesis (Section 8; Appendix R.2). They also record changes to the tests. These details help readers assess the scope of the empirical claims.

Critical weaknesses

The abstract and conclusion overstate the policy-loss result. They say every learned policy loses money, but Table 5 reports positive mean scheduling utility for logistic regression, gradient boosting, and MLP on both assets, and zero confidence-policy utility. Appendix C (p. 20) correctly limits the all-negative result to argmax. The abstract and conclusion should keep this distinction. Positive means alone do not establish significant or realized profit, but failing the joint gate also does not mean that every policy has negative utility. This distinction matters for the paper's main forecast-to-decision conclusion.

Questions for authors

What policy-specific conclusion is supported by Table 5? Please distinguish negative argmax utility, positive scheduling means, and failure of the joint gate, and state the conclusion consistently in the abstract and conclusion sections.

1 comment

Dear Reviewer ufLv,

Thanks for your effort to review this submission. Could you please confirm that your review is for submission 3300 (this paper), as the authors have concerns on its correctness? Thanks a lot!

Comments on the paper

1 comment

Dear Reviewers,

I'm writing to gently remind you that the rebuttal discussion phase deadline is approaching. Please take a moment to read the authors' rebuttal, and feel free to respond if any concerns remain that you'd like them to address. Thank you very much for your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.