acceptodds
Under review as a conference paper at ICLR 2027

When Reversible Normalization Changes What Equivariant Forecasters Can Express

Abstract

Finite-sample calibration can change which architecture a forecasting experiment recommends, even with the physical task and supervised data fixed. In Lorenz–96, position inputs (POS) and a parameter-matched equivariant-feature control (SHAM) receive the same three-rate validation search. On 128 test trajectories generated after locking selection, POS lowers horizon-32 error from 0.796 to 0.330 with four site-wise calibration trajectories. Increasing only the calibration pool to 256 reverses the mean ranking. With global normalization, the same architecture search favors SHAM and attains 0.084 using four calibration trajectories. Metric certificates preserve all four mean rankings under raw and global-scale errors. A separate five-block centering study repeats the calibration-dependent mean ranking across a sixteen-fold supervision range at fixed training budgets. An equal-dimensional construction distinguishes symmetry compatibility from statistical model choice, although its predicted supervision effect has the opposite sign in the jointly trained CNN. Gray–Scott FNO supplies a second small-pool POS benefit; CNN and long-horizon results delimit transfer. The practical consequence is that a position-input gain within one preprocessing protocol does not establish a need for position-dependent physics. Calibration choices should be controlled when interpreting architecture ablations.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
normalization, equivariance, architecture evaluation, scientific machine learning, dynamical systems, neural operators

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ya3GOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This work studies the effect of local (site-wise) standardisation with finite sample estimates changes translation equivariance compared to global standardisation when forecasting periodic systems (Lorenz-96). This effect can bias ablations experiments that compare the effect of positional encodings with equivariant controls. The results studying generalisation of the empirical findings to other systems (Allen–Cahn and Gray-Scott) are mixed.

However, arriving at this summary required an immense effort in reverse-engineering the very dense manuscript. It reads more like a compressed experimental log than a human-readable manuscript.

Critical strengths

The paper studies a practical problem in preprocessing data for equivariant forecasting. The study tries to carefully control parameter counts, random seeds, and validation sets. That means that the findings for the Lorenz-96 system are solid I think.

Critical weaknesses

Clarity and Presentation:

The manuscript is full of undefined acronyms and experimental jargon in the first 5 pages and it is immensely hard to follow and obfuscates the actual question and findings. (For example: POS, SHAM, P, G, and Mc are central to understanding Abstract and Introduction that also uses jargon like "locking selection", "metric certificates", and "five-block centering-only study". But what POS and SHAM are really only explained on Page 6.) The manuscript reads like a shorthand log without a clear scientific narrative (especially the last part of the introduction from line 048). Also the Results sections (particularly 5.1. - 5.4) are hard to parse and consist of numerical readouts and defensive statements about statistical caveats. The manuscript never cleanly explains the experimental setup, motivation, or intuition. In its current form, the paper does not communicate the assumptions, methods and findings in an understandable or verifiable way and requires a major rewrite of the first 6 pages at least.

Generalization of the findings:

In my understanding the central mathematical observation (that site-wise scaling does not commute with shifts if the statistics are not spatially constant) seems elementary. The more interesting fixed-backbone theory in Section 3.3 does not predict the supervision effect in the actual neural network experiments (Section 5.2) if I understood this correctly. This would leave a very narrow conclusion for the paper.

Questions for authors
  • I would ask how the authors plan to rewrite the first half of the paper to make it accessible so that a reader with a standard Machine Learning background can follow the motivation, definitions and experimental setting and interpret the reported results.
  • Why do you think the jointly trained CNN in Section 5.2 goes in the opposite direction of Proposition 2?
  • Why does the frozen-backbone adapter in Section 5.1 fail to reproduce the repair effect?
Reviewer_W5VTOpenReview ↗
Rating
1(−1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how finite-sample normalization affects architecture selection in equivariant forecasting. The authors show that position-wise normalization changes the represented translation action, while global normalization preserves ordinary shift equivariance. In the Lorenz-96 model, position inputs (POS) outperform a parameter-matched control (SHAM) with four calibration trajectories under position-wise normalization, but the ranking reverses with 256 trajectories. The paper provides a theoretical construction and additional experiments on other physical systems to study this effect. The main conclusion is that architecture comparisons depend on the preprocessing protocol and should not be interpreted independently of it.

Critical strengths
  • The paper addresses a relevant problem in scientific machine learning. In particular, it shows that preprocessing can change the conclusion of an architecture ablation, even when the physical task and supervised data remain fixed.

  • The mathematical analysis clearly distinguishes symmetry compatibility from prediction risk. The four-site example provides a simple illustration of why a fixed position-wise normalizer can break ordinary shift equivariance.

  • The main Lorenz-96 experiment uses matched data and training procedures, a separate test set, and a parameter-matched control. The authors also report failed runs and discuss the limitations of their conclusions.

  • The work checks whether an architecture's advantage persists under different normalization protocols before attributing the gain to position-dependent modeling.

Critical weaknesses
  • The main novelty is not sufficiently clear. The mathematical observation that normalization can break equivariance is closely related to existing work, including Scanvic et al. (2025), which is cited by the paper. The potentially novel contribution is the demonstration that changing the calibration pool size can reverse architecture rankings without adding supervised data. However, the authors should clarify how this finding advances existing work.

  • The experimental contribution is relatively modest. Lorenz-96 is a standard benchmark, and the experiments use conventional CNN (Convolutional Neural Networks) and FNO (Fourier Neural Operators) architectures. The additional reaction-diffusion experiments provide some breadth, but the results are mixed and do not establish broad generalization. Given the limited novelty of the mathematical formulation, stronger experimental evidence would be needed to demonstrate the significance of the finding.

  • The experiments do not isolate symmetry mismatch from differences in effective model capacity and optimization. Although the POS and SHAM architectures are parameter-matched and share their initial functions, their hypothesis classes and subsequent optimization differ. Therefore, the observed ranking reversal cannot be attributed specifically to symmetry mismatch.

  • The paper is not easy to follow. It introduces several experimental protocols, controls, and metrics, while the relationship between the theoretical construction and the neural network experiments is not always clear. The main contribution and the distinction between established theory and new findings should be stated more directly.

Questions for authors

1- What is the precise novelty beyond existing work on normalization and equivariance, particularly Scanvic et al. (2025), and existing positional-input methods? Which specific finding should be considered the main contribution?

2- Can the authors provide stronger experimental evidence that the calibration-dependent ranking reversal generalizes beyond the current benchmarks and architectures?

3- How can the effect of symmetry mismatch be distinguished from differences in effective capacity and optimization? Can a more directly controlled experiment support the proposed explanation?

4- Can the paper be reorganized to make the main contribution easier to follow, with a clearer separation between established mathematical results, the new theoretical construction, and the empirical findings?

1 comment

The authors did not submit a rebuttal, so the concerns raised in the review remain unaddressed.

Reviewer_2PDAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates how normalization affects the perofrmance of translation-equivariant neural nets for spatiotemporal forecasting of dynamical systems. It shows that site-wise normalization can introduce artificial position dependence, making the dynamics harder for equivariant models to learn. Experimentally, they demonstrate that positional embeddings can compensate for this mismatch when normalization stats are estimated from limited data, but their advantage disappears/reverses with larger calibration data or global normalization.

Critical strengths
  • Interesting question: The paper investigates how preprocessing choices can affect the behavior of equivariant architectures, potentially leading to misleading conclusions about which models perform best.

  • Good theoretical foundation: The authors provide a mathematical explanation of how site-wise normalization can introduce artificial position dependence, even when the underlying dynamics are translation-equivariant.

  • Controlled experiments are well executed: Although the experimental generalization is limited (see weaknesses), the main Lorenz–96 experiments are solid. The authors carefully control training conditions, use multiple seeds, and evaluate on fresh test trajectories.

Critical weaknesses
  • Unnecessarily complicated presentation: The main idea is simple (which is a good thing!), but the paper is hard to follow in my opinion. Some paragraphs are too full of details, and the paper in general introduces many experimental protocols, metrics, and theoretical details that make the central contribution difficult to follow. I would suggest to make the main text easier to follow, with a clearly defined storyline and leave some of the details to the appendix.

  • Limited practical relevance: A big concern of mine is the practical relevance of the setting used in the paper. The paper focuses on spatially homogeneous dynamical systems with site-wise normalization, but it is unclear to me how commonly this combination arises in practice. While the theoretical observation is valid, the paper would benefit from demonstrating that this issue affects commonly used forecasting settings or leads to misleading architectural conclusions in realistic settings.

  • Limited experiments results: The main experimental results (architecture-ranking reversal) is convincingly demonstrated only on Lorenz–96. Results on other systems are mixed, making it difficult to assess how widespread the phenomenon is. Moreover, all experiments use simulated physical systems, and it is unclear whether the studied effects arise in commonly used forecasting benchmarks or practical applications. I would encourage the authors to evaluate more realistic forecasting settings.

Questions for authors
  • Could the authors simplify the main text narrative and more clearly distinguish the central findings from supporting analyses?

  • Could the authors clarify how commonly site-wise normalization is used in forecasting settings where the underlying dynamics are spatially homogeneous? Are there existing benchmarks or commonly used pipelines where the identified issue might arise?

  • The architecture-ranking reversal is convincingly demonstrated on Lorenz–96, but the results on other systems are mixed. What do the authors think explains these differences?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.