acceptodds
Under review as a conference paper at ICLR 2027

How Reversible Normalization Changes Linear Forecasting

Abstract

We characterize how reversible normalization affects the function class, fitting weights, and inference-time behavior of a linear forecaster. Element-Wise Normalization (EWN) learns separate probability distributions over input lags for centering and scale estimation. These distributions are optimized by differentiating the forecasting loss through the ridge solver used to fit the final head. With a homogeneous head and shared input and output statistics, the learned scale cancels exactly at inference, even when it depends on the input. During fitting, however, squared residuals are weighted by the inverse squared scale. The learned center weights determine the affine set of attainable lag maps. Under our compressed representation, a center-weight vector outside the feature span adds one direction relative to instance normalization (IN). An intercept in normalized coordinates introduces a scale-dependent term that is generally nonlinear in the input. In a controlled heteroskedastic study, learning the scale improves estimation without changing the deployed function class. No clear advantage over IN is observed under the null condition with uniform lag weights, supporting an explanation based on residual reweighting. Both EWN variants yield lower point estimates of aggregate MSE than IN in the matched forecasting experiments, including those with homogeneous heads. Static EWN also improves on IN in the raw-lag control. EWN avoids a discrete search over suffix lengths, but the comparison with head type selected by validation establishes neither an accuracy advantage nor equivalence to trailing-window normalization.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Time-Series Forecasting, Reversible Normalization, Linear Forecasting, Element-Wise Normalization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gSkhOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the mathematical and empirical mechanisms by which reversible instance normalization alters closed-form linear time series forecasters.

Critical strengths
  1. This article attempts to explain how the most commonly used normalization methods in time series forecasting work. This is what I consider the most valuable innovation of the paper.
  2. This paper proposes an estimator-matched evaluation protocol that controls the attainable coefficient class, relative observation weights, and effective penalty. This is also what I consider another major contribution of this paper.
Critical weaknesses
  1. I believe that the method proposed in this paper does not cover all normalization methods and mainly focuses on RevIN. This is inconsistent with what is claimed in the paper's title. It makes me doubt whether the proposed method works with other normalization methods.
  2. The experiments in this paper are few and insufficient, lacking comparisons and ablation experiments on more methods. Meanwhile, the significant accuracy improvement from strong adapters is quite limited. This raises concerns about the generalizability of this method.
Questions for authors
  1. Can you distinguish the core boundaries of the normalization methods targeted in this article?
  2. Could you discuss some failure scenarios of reversible normalization? Especially in the case of distributed drift.?
Reviewer_N5naOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper analyzes how reversible normalization changes linear forecasting through observation weights, effective regularization, and the feature directions introduced by centering. Controlled experiments show that unweighted fitting often outperforms inverse-scale weighting, while learned-center gains depend on the input representation.

Critical strengths
  • Clear analytical framework. The paper carefully distinguishes the prediction function from the fitting procedure, explaining why scale cancellation at inference does not imply an unchanged estimator.
  • Well-designed controls. Effective-penalty matching, raw-lag comparisons, synthetic experiments, and results excluding Solar provide useful support for the main conclusions
Critical weaknesses
  • Limited evidence of practical scope. The weighting analysis requires normalized-coordinate loss, while the scale-cancellation result assumes a homogeneous linear head and shared encoding/decoding statistics. The paper should identify which existing forecasting pipelines satisfy these conditions and demonstrate the implications within such implementations.
  • The learned-center results need further qualification. Gains occur mainly with short compressed inputs, whereas raw-lag models achieve comparable or better absolute performance. Comparisons against rank-matched fixed or random feature directions would clarify the value of learning the added direction.
Questions for authors

Could the authors reproduce the central weighting comparison within an existing forecasting implementation using its original training objective, clarifying which conclusions remain applicable?

Reviewer_ZyKHOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper analyzes reversible instance normalization for homogeneous linear forecasting heads. It shows that the scale operation cancels at inference but changes training into a weighted ridge problem, while centering changes the feature span and induced regularization. The proposed estimator-matched protocol separates these effects with unweighted and mean-one controls, raw-lag comparisons, synthetic data, and nine real datasets.

Critical strengths
  1. The derivation exposes three effects that are usually confounded: sample weighting, effective regularization, and a change in the feature class. The mean-one weighting control and common effective-penalty grid are especially convincing.
  2. The paper uses several strong checks, including raw-lag and original-span controls, synthetic data with known noise structure, dataset-clustered intervals, and explicit negative results when the centering benefit disappears.
Critical weaknesses
  1. The empirical support is confined to closed-form linear estimators and a hand-designed compressed feature map rather than standard neural forecasting pipelines that use RevIN. The centering gain disappears with raw lags and long contexts, and the capped inner fit can leave very few distinct temporal origins for high-dimensional datasets such as Traffic. These experiments do not yet show that the claimed effects matter for practical RevIN models.
Questions for authors
  1. Can the authors repeat the unweighted-versus-mean-one comparison and the centering controls with larger temporal-origin caps on Electricity, Traffic, and Solar?
  2. Does the scale-cancellation conclusion continue to predict behavior for a standard nonlinear forecasting backbone with RevIN, even if only as a focused experiment?
  3. Can the anonymous repository link be updated to a working artifact containing the complete split and sampling records?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.