acceptodds
Under review as a conference paper at ICLR 2027

Equivariant Neural Forecasting: Pair Structure for Prediction and Analysis

Abstract

Multivariate forecasters increasingly exploit cross-variable dependencies to improve prediction, yet they typically treat these relationships as ephemeral computational pathways that dissolve after inference, leaving a divide between predictive accuracy and physical understanding. We introduce the Equivariant Neural Forecaster (ENF), an architecture that unifies coordinate consistency with explicit, disentangled relational modeling. Operating within an invertible normalization frame that guarantees joint equivariance across channel permutations and affine unit changes, ENF orthogonally decomposes channel states into global consensus and local deviation subspaces, while deploying algebraic double-centering to isolate bilateral interactions from unilateral baseline drifts. These disentangled interactions supply target-dependent messages to cross-channel attention and retain an explicit relational substrate for post-hoc analysis. By integrating equivariant sensor identity conditioning with relational subspace routing, the ENF framework achieves an overall 17.5% MSE reduction over iTransformer on four traffic benchmarks. Controlled tasks further confirm that the retained pair field renders multi-hop relational composition directly accessible to linear readouts, without requiring higher-order decoders. Beyond forecasting, diagnostic probes indicate that the learned field captures partially transferable coupling signals in chaotic circuits and guides symbolic distillation into analytical expressions that transfer to wind-tunnel measurements.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Group Equivariance, Dynamical Systems, Symbolic Regression, Cross-Variable Interactions

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6xnxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces Equivariant Neural Forecaster (ENF), an architecture that targets multivariate time-series forecasting robust to channel permutations and capable of creating artifacts containing channel-wise interactions. While the underlying motivation is interesting, the paper suffers from significant presentation and clarity issues. The text is very dense, written with a lot of jargon and synonyms, and lacks clarity. Its comprehension requires significant effort on the reader side. Furthermore, the empirical evaluation is limited and insufficient, lacks comparison to state-of-the-art methods, and fails to provide a clear explanation of the proposed forecasting mechanism and its training. In my view, a complete rewrite is required to make this work publishable, and thus, I recommend rejection.

Critical strengths
  1. The motivation behind creating an artifact that captures cross-channel interactions is compelling and appears somewhat novel. ENF appears to achieve this goal.
Critical weaknesses
  1. Clarity, presentation, and notation
    1. There is a mismatch between the title on OpenReview and the title in the PDF. Notably, the title in the PDF contains “Irreducible Representations”, which do not appear in the paper.
    2. Figure 1 is poorly formatted with overlapping text and arrows.Figure 2 is difficult to read. ENF layer structure is unclear, and a visualization of the end-to-end flow would be much more helpful. Many elements in the plot appear to be arbitrary (e.g., the grey arrow from "double center" to "channel updates”).
    3. The paper uses numerous competing synonyms and technical jargon that are never properly introduced or defined. A few examples: endpoint effects/dependencies, shared-value attention, bilateral coupling, decomposing states into a shared consensus and zero-mean deviations, shared and identity-conditioned feed-forward networks, bilinear content field.
    4. Undefined variables/notation issues.
      1. E used in B is undefined.
      2. o1o_1 and o0o_0 are never defined
      3. WOW_O, WW, WhW_h, WaW_a, WbW_b are all undefined or unclear in their context.
      4. Theorem 1: RevIN acronym hasn’t been defined. While it comes from Kim et al. 2022, which has been cited earlier, the acronym hasn’t been introduced.
      5. L356: (d64/L1) notation is undefined.
      6. L373: First-order EQNN is never defined, and we cannot deduce what this baseline represents.
  2. Methodology.
    1. The complexity of the method seems unnecessary. In L151, the authors claim that a naive neural network is coordinate-dependent. However, a simple Transformer without positional encodings and with channel-wise normalization also preserves channel permutation invariance. The definition of the “Architectural Constraint” appears to unnecessarily decouple normalization parameters from the sensor identity table, as the logical order of operations should be permuting channels first, then computing normalization statistics.
    2. The paper lacks a description of the actual forecasting mechanism. It is unclear whether the architecture is trained with a next-token or a direct-prediction objective.
  3. Experiments.
    1. Evaluation on traffic (PEMS) benchmarks is insufficient, and Q1 is not sufficiently evaluated. While the experimental setup mentions PEMS{03,04,07,08}, Electricity, Solar, Weather, ETTh1/2, ETTm1/2, and Exchange rates, Table 1 contains PEMS results only. Figure 3 contains a selection of these datasets, but not all of them. I find this quite surprising and unsatisfactory.
    2. The paper strongly emphasizes comparing its results with iTransformer, which is misguided. iTransformer has been published in 2023, and since then numerous stronger baselines have appeared. A short and non-representative list of published and highly cited methods is: CycleNet (Neurips 2024) [1], TQNet (ICML 2025) [2], DUET (KDD 2025) [3]. Moreover, in the analysis of the results, there are several errors: the authors comment on short-term forecasting vs multi-step predictions suffering from error propagation, yet no forecasting mechanism has been introduced in the paper; we also do not know whether the architecture is trained with a next-token or direct prediction objective.
    3. The authors should focus on a set of the most imporant results because the current evaluation section simply packs too many experiments and does not explain any of them sufficiently. All paragraphs just send the reader to the Appendix, and as a result, none of the four posed questions are thoroughly explored.
  4. Related work and lack of comparison to baselines that focus on forecasting with channel-wise permutation invariance. A paper published on ICLR 2026 (CPiRi [4]) appears to explore a very similar problem and has not been mentioned in the related work nor compared to in the experiments.

[1] Lin et al. 2024 “CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns”, NeurIPS https://neurips.cc/virtual/2024/poster/94391

[2] Lin et al. 2025 “Temporal Query Network for Efficient Multivariate Time Series Forecasting”, ICML https://proceedings.mlr.press/v267/lin25e.html

[3] Qiu et al. 2025 “DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting”, KDD https://doi.org/10.1145/3690624.3709325

[4] Xu et al. 2026 “CPiRi: Channel Permutation-Invariant Relational Interaction for Multivariate Time Series Forecasting”, ICLR https://arxiv.org/abs/2601.20318v1

Questions for authors
1 comment
AuthorsOpenReview ↗

Reply to Reviewer 6xnx — Original submission and presentation (1/3)

Dear Reviewer 6xnx,

Several primary grounds cited for rejection rest on verifiable factual inaccuracies regarding material already present in the submitted manuscript, alongside two methodological misconceptions regarding normalization parameters and equivariant relational modeling. We establish the factual record from the submitted PDF first, address the methodological misconceptions and ENF’s core novelty second, and present new comparisons against recent baselines (TQNet, CycleNet) and controlled ablations third.

1. Corrections to the Factual Record (Submitted Manuscript)

Nine specific items described in the review as missing, undefined, or arbitrary were explicitly defined and reported in the original submission:

  • "CPiRi has not been mentioned in the related work" — Incorrect: CPiRi (Xu et al., 2026) was explicitly discussed in Section 2 (lines 111–112) and cited in References (lines 592–593). Not evaluating CPiRi numerically is a separate point that we explicitly note in revised Section 4.1.

  • "Figure 3 contains a selection of these datasets, but not all of them... Q1 is not sufficiently evaluated" — Full benchmark already provided: The Figure 3 caption (lines 411–413) explicitly directed readers to Appendix Table 10 (lines 1089–1107), which reported the complete 13-dataset benchmark across all listed datasets. Revised Section 4.2 now links the compact summary directly to the full 13-dataset panels (Appendix Tables 15 and 22).

  • "EE used in BB is undefined" — Incorrect: The sensor identity table E∈RC×deE \in \mathbb{R}^{C \times d_e} was defined in Section 3.1 (lines 133–135 and 154–156), and its centered form Estd=PstdEE_{\mathrm{std}} = P_{\mathrm{std}}E was defined immediately inside the definition of B(Estd,D)B(E_{\mathrm{std}}, D) in Section 3.2.3 (lines 266–268).

  • "o1o_1 and o0o_0 are never defined" — Incorrect: Equation (6) in Section 3.2.3 (lines 270–275) explicitly defined (os)i=Wo∑j=1C(As)ij(vj+mij)(o_s)_i = W_o \sum_{j=1}^C (A_s)_{ij}(v_j + m_{ij}) for s∈{0,1}s \in \{0, 1\}, immediately followed by δ=Pstd(o1−o0)\delta = P_{\mathrm{std}}(o_1 - o_0).

  • "L356: (d64/L1) notation is undefined" — Incorrect: Section 4.1 (lines 342–343) explicitly defined the compact setup as (d=64,L=1)(d = 64, L = 1).

  • "L373: First-order EQNN is never defined" — Incorrect: Lines 373–374 defined First-order EQNN at that exact locus ("updates channel states via attention without second-order pair fields"), and Appendix B.1 (lines 817–819) listed all retained components.

  • "Irreducible Representations do not appear in the paper" — Incorrect: Section 3.2.1 (lines 199–204) explicitly decomposed the channel space RC\mathbb{R}^C under SCS_C into the 1D trivial and (C−1)(C-1)D standard irreducible representations, while Appendix Table 3 and Appendix A.3 (lines 760–773) explicitly distinguished those from the reducible (C−1)2(C-1)^2 pair-interaction subspace.

  • "Grey arrow from 'double center' to 'channel updates' appears arbitrary" — Route explicitly specified: Figure 2b explicitly labeled this route g, r, c -> channel updates (line 173), matching Section 3.2.2 (lines 242–244: "the single-variable terms (g,r,c)(g, r, c) are routed into channel state accumulators") and Appendix B.2 Step 1 (lines 827–829).

  • "Technical jargon never introduced or defined" — Incorrect: The cited concepts were specified in the following passages: global consensus μ=1C1⊤h\mu = \frac{1}{C}\mathbf{1}^\top h and zero-mean deviations D=PstdhD = P_{\mathrm{std}}h (1⊤D=0\mathbf{1}^\top D = 0) in §3.2.1 (lines 201–210); additive endpoint effects (g,ri,cj)(g, r_i, c_j) vs. bilateral residual R⊥=PstdRPstdR^\perp = P_{\mathrm{std}} R P_{\mathrm{std}} in Eq. (3) (§3.2.2, lines 228–236); shared-value attention (vjv_j identical across receivers vs. target-conditioned vj+mijv_j + m_{ij}) in Figure 1b and §3.2.3 (lines 248–257); bilinear content field in Eqs. (2) and (4); and shared vs. identity-conditioned FFNs in §3.2.3 (lines 279–280) and Appendix B.2 Step 4 (lines 846–848).

Targeted presentation improvements in the revision: Where presentation genuinely benefited from tightening, we expanded RevIN at first mention (§3.1), labeled projection matrices Wh,Wa,Wb,W×,WoW_h, W_a, W_b, W_\times, W_o inline (§3.2.2–3.2.3), adjusted Figure 1’s arrows to remove overlaps, placed the direct decoder and explicit MSE objective together in §3.2.3 (Eq. 8), and reorganized Section 4 around self-contained main-text tables (Tables 1–7).

Reviewer_nrSCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ENF is a channel-token forecaster that is jointly equivariant to channel permutations and to per-channel affine unit changes. It gets this from RevIN plus a sensor identity table that is permuted along with the channels. It splits channel states into a mean "consensus" part and centered "deviations". It then builds a bilinear pair field and double-centers it to get R⊥, whose messages m_ij are added to the attention values. An exact factorization means the C²·d_r pair tensor is never stored. It reports a 17.5% MSE reduction over iTransformer on PEMS03/04/07/08, plus a 13-dataset study, synthetic relational probes, a Rössler coupling readout and symbolic distillation.

Critical strengths
  1. The design is principled: a group-theoretic formulation, double-centering to separate additive endpoint effects from interactions, and proofs. The factorization in Prop. 1 is a neat practical trick that removes O(C²d_r) memory and gives about a 14× step speedup at C=256.
  2. Gains on PEMS are consistent under matched capacity over three seeds. The C-scaling study (Table 9) shows the advantage growing with channel count.
  3. The appendix is thorough. The controlled probes show that R⊥ carries linearly decodable relational information that first-order node states lack.
Critical weaknesses
  1. The forecasting gains come mainly from the learned sensor identity rather than the new components. In the staged comparison of Table 5, adding identities to iTransformer accounts for a 12.4% reduction, while the consensus/deviation routing and the relational configuration add only 2.0% and 3.9%. Against iTransformer with learned identities, the PEMS gain therefore shrinks from 17.5% to about 6%. Table 6 further shows that removing the interaction message or zeroing R⊥ changes MSE by at most 1.6%, which the paper itself describes as a marginal impact.
  2. The equivariance guarantee adds little beyond existing designs. The affine part of Theorem 1 relies on RevIN, which is disabled on Solar, and standard self-attention over channel tokens is already permutation equivariant. Because the identity table is permuted together with the channels, the joint law places little constraint on what the model learns. The paper verifies equivariance numerically, but no experiment shows that it improves forecasting or transfer.
  3. The physical-discovery claims go beyond the evidence. The coupling readout on the Rössler circuits reaches only R² = 0.0996, compared with 0.0808 for the additive baseline. For symbolic distillation, the pair field is only used to select variable pairs, the formulas are fitted to model forecasts, and the paper acknowledges that static rank correlation is an equally effective screen. The claim of "genuine physical fidelity" in the conclusion is therefore not well supported.
Questions for authors
  1. Could iTransformer with learned identities be reported on all datasets? STID is also listed as a baseline, but no STID results appear in the paper.
  2. Is there a setting, such as unseen channel orderings or units, where the equivariance gives a measurable benefit over iTransformer with RevIN?
Reviewer_L1fYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the Equivariant Neural Forecaster (ENF) for multivariate time series forecasting and relational analysis. The model separates channel representations into global consensus and local deviations, constructs an explicit pair field, and applies double-centering to remove additive endpoint components. The resulting interaction features are incorporated into cross-channel attention and retained for downstream analysis. The paper establishes joint coordinate equivariance and an exact factorization of the interaction computation, and evaluates the framework on forecasting benchmarks, controlled relation tasks, and physical systems.

Critical strengths
  1. The paper addresses a meaningful problem: how to improve multivariate forecasting while making cross-variable representations available for subsequent analysis. The distinction between forecasting performance and the information accessible in learned representations is useful.
  2. The mathematical construction is coherent. The joint transformation of observations and sensor identities is specified, and the factorization avoids explicitly materializing the full pair-feature tensor during deterministic prediction.
  3. The evaluation covers multiple datasets, forecasting horizons, and controlled tasks. The authors also report ablations and negative findings rather than presenting uniformly favorable results.
Critical weaknesses
  1. The paper does not sufficiently demonstrate that explicit relational messages are a major source of the forecasting improvements. Table 5 suggests that much of the improvement comes from variable identity information, while the ablation results in Tables 6 and 17 show that disabling interaction messages generally has little impact on forecasting performance. Therefore, the effectiveness of the full model does not directly establish the substantive contribution of the proposed second-order relational message mechanism, and the contributions of individual components remain insufficiently disentangled.

  2. The physical interpretation of the double-centered relational field is not sufficiently justified. Double-centering only removes additive endpoint terms in the chosen feature space and does not guarantee that the residual representation corresponds to genuine physical interactions. The physical readout in Table 28 relies on additional supervision and exhibits limited explanatory power. Moreover, the double-centered representation does not outperform the full relational field. These results suggest that the representation contains some physical interaction information, but they are insufficient to establish that double-centering achieves the physical relation disentanglement emphasized by the paper.

  3. The paper does not sufficiently establish the efficiency of the complete model relative to existing baselines. Table 13 shows that dense ENF incurs substantial computational and memory overhead compared with iTransformer, while Table 14 demonstrates the efficiency gains of factorization relative to the dense ENF implementation. However, a direct comparison between the factorized ENF and the baseline under matched conditions is missing. Consequently, it remains unclear whether the proposed factorization yields a competitive accuracy-efficiency trade-off in the complete model.

Questions for authors
  1. What forecasting benefit can be attributed specifically to the relational message mechanism, independently of variable identities and other configuration changes, given the ablations in Tables 5, 6, and 17?

  2. What supports interpreting the double-centered field as physically disentangled interactions, rather than a useful statistical representation, given the supervised readout results in Table 28?

  3. How does the complete factorized ENF compare with iTransformer in accuracy and computational cost under matched conditions, beyond the dense-versus-factorized comparison in Tables 13 and 14?

Comments on the paper

1 comment

Dear Reviewers,

The authors have now submitted their responses. Please review them and indicate whether your concerns have been addressed. Please also update your scores if the responses have changed your assessment.

Thank you.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.