acceptodds
Withdrawn from ICLR 2027

When Does RevIN Hurt? A Statistical Analysis of Reversible Normalization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reversible Instance Normalization (RevIN) has become widely adopted in time series forecasting. However, we show that mean-only normalization can match or outperform full RevIN across a substantial fraction of configurations. To understand this counterintuitive phenomenon, we develop a statistical framework that characterizes the effect of normalization through two competing factors, a structural floor induced by information loss and a transfer distance arising from shifts in the learned mapping between training and deployment. This formulation reveals an inherent tradeoff: normalization can reduce transfer distance by discarding distributional information, but the same information loss introduces a structural floor. Based on this framework, we derive a closed-form characterization of the performance gap between mean normalization and raw data, yielding an exact criterion for mean removal. For variance scaling, whose nonlinear effect precludes an exact characterization, we introduce proxy statistics that capture the corresponding structural floor and transfer distance, and validate their causal roles through controlled experiments on real datasets. Together, these results provide a unified theoretical characterization of RevIN and criteria for determining when its two components are effective.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Reversible Instance Normalization, Normalization, Deployment Risk, Non-stationarity

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_sfXuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates when mean removal and variance scaling in reversible instance normalization help or hurt time series forecasting. It studies raw inputs, mean-only normalization, and full normalization through a framework separating structural approximation cost from the discrepancy between a trained predictor and a deployment oracle. The paper derives a least-squares decomposition of level prediction error and claims an exact expression comparing raw and mean-only prediction. For variance scaling, it develops a scale model and proposes diagnostic statistics for information loss and distribution shift. Experiments across several forecasting architectures and datasets, together with synthetic perturbations, are used to support the analysis.

Critical strengths
  1. The paper studies mean removal and variance scaling separately across multiple forecasting architectures and includes controlled perturbation experiments. The least-squares decomposition in Theorem 5.1 is a sensible diagnostic starting point. These components could support an informative empirical study even if the stronger claimed characterization needs revision.
Critical weaknesses
  1. lack of novelty and soundness, analyses are not convincing enough for publication. the contributions are mostly around an analyzing framework, which does not provide futher insight regarding improvement on revin and future directions.
  2. Mean-only normalization does not generally pin the forecast mean to the historical mean, invalidating the stated specialization behind Theorem 5.2
  3. The amplitude-coupling proxy does not distinguish recoverable scale dependence from information lost by normalization.
Questions for authors
  1. Do the implemented mean-only predictors explicitly enforce 1H⊤g(x~)=0\mathbf 1_H^\top g(\widetilde x)=0 for every input? If so, where is that operation defined? If not, how is the counterexample above reconciled with Theorem 5.2, and was the empirical comparison made against a separately trained mean-only model or against a predictor whose level was forced to the historical mean?

  2. What makes rtr+rter_{\mathrm{tr}}+r_{\mathrm{te}} informative about structural information loss beyond strong amplitude dependence alone? How does the proposed diagnostic behave in the constant-ρ\rho case where the theory gives zero structural loss, and what evidence connects it to the variation of ρ(σ)\rho(\sigma)?

  3. Why is the noisy Full predictor represented as σ(1+ξ)h(ϕ)\sigma(1+\xi)h(\phi) rather than σ(1+ξ)h(ϕ/(1+ξ))\sigma(1+\xi)h(\phi/(1+\xi))? Is there an assumption on the backbone or on the definition of ϕ\phi that makes these equivalent, and does that assumption hold for the evaluated models?

  4. Please reconcile the stated E[ξ]=O(L−2)\mathbb E[\xi]=O(L^{-2}) with the −(κ4−1)/(8L)-(\kappa_4-1)/(8L) term from the displayed Taylor expansion. After retaining the bias and re-optimizing the readout if necessary, which coefficients and remainder orders in Eq. (7) remain unchanged?

  5. How much of the synthetic performance trend can be explained by the simultaneous changes in δs\delta_s and δρ\delta_\rho shown in Table 2? Can the existing intervention results be matched or adjusted on those quantities, or can a small control hold them approximately fixed while varying the intended factor?

Reviewer_uNxHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper provides a systematic analysis of Reversible Instance Normalization (RevIN), examining when its mean-removal and variance-scaling steps help or hurt forecasting. The authors frame this as a trade-off between a structural floor, information lost by normalizing and a transfer distance, train–test mismatch that normalizing removes. They compare mean-only normalization (MO) with full RevIN, present a closed-form characterization of the level gap, and study variance scaling through proxy statistics and synthetic intervention. The authors report that MO matches or outperforms RevIN in a large portion of configurations across six backbones, seven datasets, and four horizons, with outcomes varying across configurations.

Critical strengths
  • The MO-RevIN comparison covers six backbones, seven datasets, and four horizons under a common protocol.
  • The level-shape decomposition provides a useful diagnostic, with level error explaining most of the gap between raw and mean-only inputs in the evaluated settings.
  • Section 6.2 shows how near-flat windows with non-zero means can produce extreme values after variance-only scaling, helping explain its observed poor performance.
  • Proposition 6.1 predicts that the scale-estimation cost decreases with longer lookback windows, and the lookback experiments show a qualitatively consistent trend.
Critical weaknesses
  • The contribution is limited relative to the claims. The result covers only raw vs. mean-only inputs, while the motivating question of mean-only vs. full RevIN is answered only by an empirical rule supported by a single dataset. The paper therefore does not deliver the unified characterization or the criteria for when each component is effective that the abstract promises.
  • The level-gap result assumes that the mean-only model's forecast level equals the input window mean, but input centering does not force the backbone's output to have zero mean. Table 4 shows substantial discrepancies between estimated and measured gaps. The formula also requires test labels, which limits its use for choosing a normalization in advance.
  • Proposition 6.1 implies that variance scaling loses information only when the ratio between future amplitude and window scale varies across windows. A strong but constant ratio causes no loss, because the scale is restored at the output. The proposed proxy, however, measures the strength of this relationship rather than its variation. A high proxy value does not necessarily indicate a high cost.
  • The synthetic interventions do not fully separate the proposed factors. The kurtosis sweep also changes the scale drift, and the coupling intervention changes the coupling drift despite Table 2 describing it as fixed.
  • MO is competitive in many configurations, but gains vary across datasets and many comparisons are close or tied. Without seed counts or variability estimates, the reliability of these differences is difficult to assess.
Questions for authors
  • How large is the level offset the trained mean-only model adds to the window mean, and does accounting for it reconcile the discrepancies in Table 4? Can the level-gap criterion be computed without test labels?
  • How does the proposed coupling proxy track the amplitude-information loss in Proposition 6.1, given that a large constant coupling can produce zero amplitude-information loss?
  • Can you run the kurtosis and coupling interventions for a representative configuration with the relevant drift quantities held fixed, or quantify how much their residual changes affect the results?
  • How many seeds underlie Table 5, and what is the run-to-run variability of the reported errors?
Reviewer_XiF4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks when the two parts of Reversible Instance Normalization (RevIN), mean removal and variance scaling, help or hurt forecasting. It finds that mean-only normalization matches or beats full RevIN in many model-dataset setups (Figure 1, Table 5). It splits test risk into Bayes noise, a "structural floor" and a "transfer distance" (Theorem 4.1), and derives an exact identity for the gap between a raw model and a mean-only model (Theorems 5.1-5.2). For variance scaling it gives an approximate gap formula (Proposition 6.1) with proxy statistics (ρs\rho_s, δs\delta_s, δρ\delta_\rho, κ^4\hat\kappa_4). It tests these with synthetic interventions on ETTh1 (Figures 6 and 8, Table 2) and proposes an empirical rule for picking Full over Mean.

Critical strengths
  1. Figure 1 and Table 5 show, across six backbones and seven datasets, that mean-only and full RevIN trade wins. Traffic favours Full, and the ETT sets often favour Mean. This is a useful finding against using RevIN as a fixed default. I was glad to see that it's honest: it shows the Traffic cases where Full wins, not only the good ones.

  2. The level part has exact, checkable theory: the Raw-versus-Mean gap is derived as an identity. I checked the proofs of Theorem C.1, Theorem 5.1 (Appendix E.1) and Theorem 5.2 (Appendix E.4) and found no error. Figures 7 and 10 compare estimated and measured gaps for four backbones (Table 4 lists the values for two of them). That test could have failed. If the paper is accepted, this is the part I'd keep.

Critical weaknesses
  1. The Mean-versus-Full rule is not supported by the evidence given. The abstract promises criteria for when the two parts work, and the introduction (L92-93) promises an empirical basis for picking a normalization strategy. Section 7.2 (L469-474) says Full helps most when ρs\rho_s and κ^4\hat\kappa_4 are both fairly small. But the Conclusion (L479-481) says the choice "remains unresolved", because Full's clear advantage shows up only on Traffic. In Table 3, Traffic has the largest κ^4\hat\kappa_4 of the seven datasets (5.89, against 1.81-3.02 elsewhere). So the one dataset that supports the rule meets only the ρs\rho_s half of it. Also, ρs\rho_s is built from the test-set slope. L403-406 say the test set is used for evaluation, and the paper doesn't evaluate a predictor computed from training or validation data only. A rule that can't be used before seeing test data, and is checked on one dataset, is not yet a criterion for choosing. The link between the proxies and the formula is also loose. Appendix E.5 gives a term proportional to Varw(ρ)\mathrm{Var}_w(\rho), which is zero when the coupling ρ\rho is constant. But ρs\rho_s is a sum of regression slopes, and for a constant ρ\rho the slope equals ρ\rho, not zero. I couldn't find a derivation that connects the two. Unless the rule is shown to predict the winner beyond Traffic from training or validation statistics, the abstract and introduction should be narrowed to what is shown: the exact level-gap identity and the Raw-versus-Mean finding.

  2. The "causal" reading of κ^4\hat\kappa_4 is partly confounded, and the evidence base is narrow. In Table 2(b), as the kurtosis intervention grows, κ^4,te\hat\kappa_{4,te} rises from 3.24 to 20.33. But δs\delta_s falls from +0.588 to +0.161, a drop of about 70 percent that the caption calls modest, and Section 7.2 reports that the gap grows with δs\delta_s. To be fair, from p=1.0p=1.0 to 1.51.5, δs\delta_s is flat (0.588, 0.587) while the gap in Figure 8 (middle) still falls, so part of the effect may be real. But most of the fall happens where δs\delta_s also falls. The test slope rter_{te} moves too, from 0.074 to about 0.10, even though the caption says the coupling drift is held fixed. The intervention evidence comes from ETTh1 with two backbones in the main text (plus ETTh1 and ETTm1 with two other backbones in Figure 11). It uses L=96L=96, except for the synthetic lookback sweep in Figure 8 (right). It removes the affine layer (L406-408) and uses one mostly shared hyperparameter set. Since the estimation-noise term of Proposition 6.1 scales as 1/L1/L, it is larger at 96 than at 336 or 512. Table 5 and Figure 1 report single numbers. I couldn't find seeds, standard deviations or a significance test, many entries differ in the third decimal, and the win rate in Figure 1(H) doesn't say how ties are counted. So the "causal roles" wording (abstract, L23-24) is stronger than the evidence.

A few smaller points. The Figure 1 caption refers to "Table 6", which I couldn't find (the tables end at 5). The symbol ϵ\epsilon means forecast noise in Proposition 6.1 but the innovation in the definition of κ^4\hat\kappa_4 (L289-296). I couldn't find how κ^4\hat\kappa_4 is estimated on the real datasets of Table 3; Appendix B.2 describes a rolling-standard-deviation whitening only for the synthetic intervention. Several figures have axis text too small to read at print size.

Questions for authors
  1. For each of the seven datasets and each backbone, what is the sign of the Mean-versus-Full gap, next to (ρs,δs,δρ,κ^4)(\rho_s, \delta_s, \delta_\rho, \hat\kappa_4) computed from training or validation data only? This would show whether the rule holds beyond Traffic.
  2. In Table 2(b), what is the gap when δs\delta_s and rter_{te} are held fixed and κ^4\hat\kappa_4 varies? Or what does a regression on both give?
  3. Do the Mean-versus-Raw and Mean-versus-Full conclusions hold at L=336L=336 or 512512 on the real datasets, and when the affine layer is kept?
  4. How much do Table 5 and Figure 1(H) change across seeds, and how are ties counted in the win rate?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.