acceptodds
Under review as a conference paper at ICLR 2027

State-Conditioned Invertible Channel Dependencies for Multivariate Time Series Forecasting

Abstract

Multivariate time series (MTS) forecasting requires jointly capturing inter-series dependencies and intra-series temporal evolutions. Although inverted Transformers (e.g., iTransformer) and Graph Neural Networks can capture arbitrary pairwise inter-channel dependencies, they typically suffer from unconstrained pairwise attention maps that are prone to overfitting spurious cross-channel correlations. Furthermore, aggregating multiple channels into a weighted sum risks destructive signal cancellation and over-smoothing across distinct channels. To address these limitations, we propose a State-conditioned Invertible Channel Dependencies Network (SICDNet) that comprises two dedicated components: Structured Invertible Channel Mixer (SICM) for inter-series correlation modeling and the Spectral-Gated Modulator (SGM) for intra-series feature extraction. SICM establishes pairwise channel interactions on latent features using a dyadic pairing schedule, which adopts a multi-stage, multi-round cascade with cyclic node-index shifts to cover more cross-channel pairs. Specifically, SICM groups the latent representations and executes Givens rotations within each paired channels of each group, and the rotation angles are conditioned dynamically based on pairwise energy. SICM is energy-preserving and mathematically invertible, enabling the extraction of robust inter-channel correlations. Complementarily, SGM combines static harmonic priors with data-dependent dynamic spectral gating and dual-domain gated fusion, adaptively capturing global periodic patterns. Theoretical analysis demonstrates that SICDNet reduces the inter-series mixing complexity from to while maintaining channel-independent linear scaling in intra-series spectral processing, outperforming dense attention and graph estimation architectures.

Primary area:
learning on time series and dynamical systems
Keywords:
State-conditioned Invertible Channel Dependencies, Inter-Series Dependency, Multivariate Time Series Forecasting, Energy Preservation.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_t5a1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses cross-channel dependency modeling in multivariate time series forecasting and proposes SICDNet, which consists of two modules. (1) SICM replaces dense attention and adaptive graph-based methods with a dyadic, multi-stage/multi-round cascade of grouped Givens rotations, including cyclic shifts. It performs orthogonal, energy-preserving pairwise mixing along the variable dimension of inverted tokens. The rotation angles are dynamically generated by a small MLP from pairwise energies that are invariant to the rotations themselves. This enables exact inversion without caching intermediate angles. The computational complexity is reduced from O(N2)O(N^2) to O(R⋅Nlog⁡N)O(R\cdot N\log N). (2) SGM applies a Type-II discrete cosine transform (DCT-II) along the latent dimension, combines a learnable static frequency response with data-dependent gating, and then performs multiplicative fusion between the frequency and feature domains. The experiments cover 13 datasets and 9 baselines. The authors also evaluate the generalizability of SICM by integrating it into DUET, iTransformer, SOFTS, and VCformer; conduct identity-control causal intervention analyses; visualize rotation angles; and investigate hyperparameter sensitivity.

Critical strengths

1: The design of SICM is interesting and technically well motivated. The main idea is to generate rotation angles from pairwise energies that remain unchanged under the rotations. This allows SICM to perform input-dependent orthogonal mixing while retaining exact invertibility without storing intermediate angles. The combination of dynamic conditioning and invertible transformations is a useful design choice for time-series channel modeling. The identity-control experiment also provides evidence that cross-channel mixing contributes to the results: setting the rotation angles to zero increases MAE, and the reported bootstrap confidence interval remains above zero. 2: The experiments provide useful evidence that SICM can work with different forecasting architectures. The authors integrate the module into DUET, iTransformer, SOFTS, and VCformer, rather than evaluating it only within their own model. Using the official hyperparameters and a consistent experimental setup makes these comparisons more informative. The paper also reports cases where the improvements are unstable, particularly on Solar and Traffic, instead of presenting the results as uniformly positive. This makes the evaluation more convincing.

Critical weaknesses

1: The relationship to existing methods needs to be clarified. SICM uses a dyadic cascade of Givens rotations, which is closely related to butterfly-style factorizations of structured linear transformations (e.g., Dao et al., ICML 2019). Its orthogonality and invertibility also connect it to established invertible neural networks. Meanwhile, SGM shares similarities with frequency-domain approaches such as FreTS, FNet, and GFNet. The paper does not discuss these connections, making it difficult to determine what is genuinely new. The main distinction appears to be the use of rotation-invariant pairwise energies to generate input-dependent rotation angles. The authors should explain more clearly how this differs from existing methods and what additional benefits it provides. 2: The practical value of invertibility is not demonstrated, and some performance gains are small. Although exact inversion is emphasized as a key property of SICM, the experiments do not show how it can be used in practice, for example, to reduce activation memory during backpropagation. Without such results, it is unclear how much this property contributes beyond the mathematical design. In addition, several reported improvements are small relative to the reported variation across runs. SICDNet also performs worse than iTransformer on Traffic in terms of MSE. These results suggest that the advantage is not consistent across datasets, so the claims about its overall superiority should be more carefully qualified.

Questions for authors

1: Table 3 refers to validation-set averages, whereas Table 10 appears to report test-set results. Could the authors clarify which evaluation split was used? In addition, how were ties handled when calculating the first-place counts in Table 8 2: Section 3.3 describes the DCT as operating along the temporal dimension and motivates it through the concentration of seasonal patterns and low-frequency trends. However, the implementation appears to apply the transform along the latent feature dimension. Could the authors clarify this discrepancy and explain the motivation for applying the DCT along that dimension? 3: SICM's dyadic cascade of Givens rotations appears closely related to butterfly factorizations, while its invertibility connects it to existing reversible neural networks. Could the authors clarify the main technical differences from these approaches? In particular, beyond generating rotation angles from rotation-invariant pairwise energies, what does SICM offer that existing structured transformations do not? 4: Section 3.3 describes the DCT as operating along the temporal dimension and motivates it through the concentration of seasonal patterns and low-frequency trends. However, the implementation appears to apply the transform along the latent feature dimension. Could the authors clarify this discrepancy and explain the motivation for applying the DCT along that dimension?

Reviewer_AQCNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SICDNet for multivariate time series forecasting, combining a Structured Invertible Channel Mixer (SICM) with a Spectral-Gated Modulator (SGM). SICM mixes variable tokens through a multi-scale cascade of dyadic, group-wise Givens rotations. Its angles depend on rotation-invariant pair-group energies, enabling cache-free inversion in exact arithmetic. SGM applies a DCT along each token’s latent feature dimension, adaptively modulates the spectral coefficients, and multiplicatively fuses the reconstructed features with a feature-domain projection.

Experiments cover nine long-term and four short-term forecasting benchmarks, with plug-in evaluations in DUET, iTransformer, SOFTS, and VCformer. Ablations, an inference-time intervention that sets all rotation angles to zero, and angle diagnostics further examine the mixer’s contribution and behavior.

Critical strengths

A principled structure for adaptive channel mixing. SICM supports signed cross-variable mixing through group-wise Givens rotations while preserving feature energy. The small orthogonality residuals in Figure 8 support the numerical implementation of this explicit structural constraint.

Evidence beyond the proposed backbone. SICM is evaluated within four existing architectures using their released hyperparameters. Improvements such as VCformer’s average Electricity MSE reduction from 0.186 to 0.169 (Table 2) strengthen the case for SICM as a useful component beyond the complete SICDNet architecture.

Critical weaknesses

The added value of the proposed mechanisms needs stronger support. SICM’s dyadic rotations are structurally related to FFT-style unitary networks (Jing et al., 2017) and learned butterfly transforms (Dao et al., 2019). Although Table 3 ablates energy conditioning, SICM’s forecasting advantage over these parameterizations remains unclear. The experiments also do not identify a specific benefit of invertibility beyond energy preservation and structured mixing. For SGM, the gate and filter ablations do not establish an advantage over standard nonlinear token refinement. Its temporal-frequency interpretation also requires justification because the DCT acts on learned latent coordinates without a prescribed temporal ordering.

The empirical comparisons need clearer controls and reporting. The dataset-specific loss choices, Huber thresholds, and tuning budgets for the main comparisons in Table 1 should be specified. The stated standard-deviation bounds should be supplemented with the number of independent runs and per-setting uncertainty to assess small improvements. Tables 3 and 10 provide the corresponding ablation results as validation and test performance, respectively; this requires clarification. The regressions across all three backbones evaluated on Traffic (Table 2) also limit support for the conclusion’s broad generality claim.

Reference:

Li Jing, Yichen Shen, Tena Dubcek, John Peurifoy, Scott Skirlo, Yann LeCun, Max Tegmark, Marin Soljačić. Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs. Proceedings of the 34th International Conference on Machine Learning, 2017, 70,1733-1741.

Tri Dao, Albert Gu, Matthew Eichhorn, Atri Rudra, Christopher Re. Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations. Proceedings of the 36th International Conference on Machine Learning, 2019, 97, 1517-1527.

Questions for authors

[1] How does SICM compare with established rotation or butterfly parameterizations, and what evidence identifies the forecasting contributions of invariant state conditioning and invertibility?

[2] How does SGM compare with a standard feed-forward module of similar parameter count? What supports the temporal-frequency interpretation of its DCT along learned latent features?

[3] Which training loss and, where applicable, Huber threshold were used for each dataset? Were training objectives and tuning budgets comparable across the methods in Table 1?

[4] Were the Table 3 ablations evaluated on validation or test data, and how were hyperparameters selected? How many independent runs were performed, and are the reported gains and Traffic regressions consistent across runs?

Reviewer_A8TxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SICDNet for multivariate time series forecasting. It combines a structured invertible channel mixer (SICM) for cross-variable interaction with a spectral-gated modulator (SGM) for within-variable feature refinement. SICM performs group-wise Givens rotations according to a multistage dyadic pairing schedule. Its rotation angles are conditioned on pair energies that remain invariant under the corresponding rotation, allowing the mixer to be inverted without storing the forward angles. The authors evaluate the model on thirteen forecasting datasets and investigate SICM as a replacement for interaction modules in several existing architectures.

Critical strengths
  1. The proposed mixer offers a coherent way to combine input-dependent cross-variable computation with exact invertibility. Conditioning the angles on preserved pair energies allows the same angles to be recomputed during inversion.

  2. The structured pairing design avoids constructing a dense channel-attention matrix and provides a practically relevant alternative for multivariate forecasting with many variables.

  3. The evaluation includes multiple datasets, backbone-replacement experiments, and component ablations, providing evidence that the proposed framework can be integrated into different forecasting architectures.

Critical weaknesses
  1. The proposed inter-variable dependency modeling relies on the rotation mechanism, whose angles are determined solely by joint energy and a stage-specific embedding. This design appears to be primarily motivated by the desired invertibility and energy-preservation properties, but the paper does not sufficiently explain why this energy-based conditioning can effectively model inter-variable dependencies.

  2. The proposed mixer restricts direct interactions to pairs specified by a dyadic schedule, with cyclic shifts used to expand interaction coverage. However, the paper does not sufficiently explain why this restricted interaction pattern is adequate or why the omitted pairwise interactions can be safely excluded. Moreover, the sequential execution of multiple stages may introduce practical inference latency despite the reduced theoretical complexity. The paper lacks direct evidence demonstrating the advantages of this structured mixing strategy over more parallelizable attention-based approaches.

  3. The role of SGM in the overall framework is unclear. While SICM focuses on inter-variable dependency modeling, SGM introduces an additional spectral-gating module for intra-variable modeling. The paper does not sufficiently explain how this module relates to the central contribution, making the overall methodological motivation less coherent and the distinct contributions of the two modules unclear.

Questions for authors
  1. Why is joint pair energy an effective conditioning statistic for predictive inter-variable dependencies, beyond enabling invertibility and energy preservation?

  2. What supports the adequacy of the finite-round dyadic interaction schedule, including its practical accuracy-efficiency trade-off relative to attention-based mixing?

  3. How does SGM complement SICM, and what evidence establishes its distinct contribution beyond ordinary within-variable feature refinement?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.