acceptodds
Withdrawn from ICLR 2027

Reprogamming Cross-Channel Interaction for Noise-Resilient Time Series Forecasting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Transformer-based models have achieved remarkable performance in multivariate time series forecasting, benefiting from the attention mechanism. However, current Transformer-based forecasting paradigms still suffer from channel noise to the point that even perturbations from a single channel can significantly degrade overall performance. This vulnerability arises from three factors: noise propagation caused by token-to-token attention (*mechanism level*), loss of global information (*representation level*), and the lack of cross-channel interaction regularization (*learning level*). To address these challenges, we propose three innovations: *(i)* reprogramming attention into an MLP-based 'Contextual-weighted Interaction Block' (CoIn), *(ii)* introducing a learnable 'Global Token' to preserve global information, and *(iii)* employing 'Channel Decoupling Loss' to regularize cross-channel interaction. In particular, we renovate the traditional attention, which inevitably causes noise propagation between channels, into CoIn, which suppresses channel noise by reprogramming information flow in attention. A learnable Global Token is employed to aggregate and redistribute global information. We also introduce 'Channel Decoupling Loss', which regularizes cross-channel interactions during the learning process. Experiments on benchmark datasets show that these tools improve robustness to channel noise and outperform Transformer-based models in forecasting accuracy and efficiency. Code is available at this [**Anonymous Repo**](https://anonymous.4open.science/r/ResNo-laoda24).

Primary area:
learning on time series and dynamical systems
Keywords:
Channel interaction, Efficiency, Deep learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Hg1NOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ResNo for noise-resilient multivariate forecasting, combining a linear-complexity Contextual-weighted Interaction block (CoIn), a global token, and a Channel Decoupling Loss (CDL). The results on high-dimensional datasets are competitive, and efficient cross-channel modeling is an important problem.

Critical strengths

The results on high-dimensional datasets are competitive, and efficient cross-channel modeling is an important problem.

Critical weaknesses

The main concern is that CoIn does not clearly prevent noise propagation. All channels are first aggregated into a shared context, so a noisy channel can still contaminate this context and affect every output. The receiving gate does not identify which source contributed the noise. Thus, CoIn is better understood as low-rank global aggregation than as a mechanism that isolates noisy channels.

The novelty relative to SOFTS/STAR is also unclear. Both aggregate channels into a global representation and redistribute it with linear complexity, yet SOFTS is not included as a baseline. Moreover, CDL subtracts randomly paired channels from both inputs and targets; this appears closer to structured data augmentation than channel decoupling.

Finally, the noise evaluation is limited to Gaussian perturbations applied to one fixed channel on one dataset. Evaluations with unseen test-time corruptions, different corrupted channels and noise types, and errors measured only on clean channels are necessary to substantiate the noise-resilience claim.

Overall, the method is efficient and empirically promising, but its novelty and central mechanism are not sufficiently established.

Questions for authors

How does CoIn prevent a noisy channel from contaminating the shared context, and how is it fundamentally different from the centralized aggregate-and-redistribute mechanism in SOFTS/STAR? Please provide a direct comparison and report the forecasting error on clean channels when unseen channels are corrupted at test time.

Reviewer_xpGSOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The central problem addressed in this paper is that Transformer-based models suffer from (1) cross-channel noise propagation, (2) global context distortion as channel noise harms token interactions, (3) no regularization on cross-channel interactions and being prone to fitting spurious cross-channel correlations. To tackle it, the authors introduce ResNo a model comprising of three innovations. It replaces attention with contextual-weighted interaction block (CoIn) aiming at suppressing the influence of noisy channels; introduces a learnable global token, akin to register tokens known from ViTs; adds channel decoupling loss, which is an extra loss term that computes MSE over a perturbation of the example and requires a second forward pass. The paper shows competitive long-horizon forecasting on 9 datasets and better robustness to gaussian noise injection (CoIn vs Attention).

The novelty of the proposed model is limited. It replaces self-attention with a variant of linear attention, adds register tokens, and add augumentations via subtracting a randomly selected channel from another. While the central claims pertain to channel-noise impacting the performance of the multivariate forecasting models, this aspect is not thoroughly evaluated in the paper and the only experiment with extra noise injections deals with gaussian N(0, 1) noise. Empirical evaluation is very limited and has several flows outlined in the weaknesses section. The clarity of writing is low and there are several instances of incorrect claims, questionable citations and state-of-the-art over-claiming. My recommendation for the authors is to improve writing, literature positioning, and empirical evaluations, and for the paper I recommend rejection.

Critical strengths
  1. ResNo seems to achieve competitive results even though it replaces the attention mechanism with a linear variant of it.
  2. Ablations show that CoIn’s performance degrades less drastically than iTransformer’s with increased channel noise.
  3. Code is available online.
Critical weaknesses

Major:

  1. State-of-the-art claims are unsupported and the empirical evaluation is limited.

    1. First, the experimental evaluation (Table 2) is very limited as only one lookback length (96) was evaluated and only long-horizon tasks and datasets have been considered. Second, the latest baseline model is from 2024, which in the rapidly developing area as time series is unacceptable, especially since the authors cite many newer methods earlier in the paper. I decided to randomly check some of papers cited earlier in the paper, and for instance, CycleNet (Neurips 2024)’s reported numbers in their paper would outperform ResNo on Weather, ECL and Solar datasets. Results from the cited SOFTS (Neurips 2024) also seem very similar.
    2. L:370: “The former represents the most advanced Channel-Independent approach, while the latter is an outstanding Channel-Dependent multivariate time series forecaster.” This sentence refers to PatchTST and iTransformer, and is a scientifically invalid claim. For instance, the cited CycleNet (Neurips 2024) is a channel-independent method which outperforms PatchTST, and CycleNet’s followup TQNet (ICML 2025) is a channel-dependent version which outperforms both PatchTST and iTransformer.
    3. It is paramount to run empirical evaluations across a few random seeds and specify the hyperparameter tuning setup.
    4. The only ablation on noise injection considers standard gaussian noise, which is insufficient for a paper which focuses on time series channel noise.
  2. L:465 Depth Scalability of CoIn and Fig 6b. It should not be the case that deeper models start diverging while adding more blocks (e.g., see ResNet). That points to an issue with the training setup, for instance, learning rate, initialization, and early stopping. If authors believe that this behavior is universal then an explanation or analysis should follow and explain why noise would be the reason for this behavior in PatchTST and iTransformer.

  3. Channel decoupling loss randomly subtracts selected channel-wise signals, which is an interesting choice because it substantially changes the sample, creating possibly out-of-distribution examples. In my view this is an augumentation combined with an extra loss term because it alters the training example and requires two separate forward passes for a single gradient descent step. Moreover, it is not clear how this loss is exactly applied and the index sampling procedure is not sufficiently explained (I can’t parse L236-240).

Minor/writing:

  1. Abstract has duplicated enumeration of the three contributions.
  2. All three citations in L045 to support the claim that “Transformer models with self-attention still struggle to manage channel noise, a common issue in real-world multivariate time series data” seem to be rather unrelated, and do not consider time series at all.
  3. L130: “But in multivariate time series analysis, each series possesses unique semantic information that cannot be inferred from other variates (Granger, 1969).” - this claim is incorrect as is. No constraints on the task were introduced and it is entirely plausible that somme channels contain fully redundant information or can be reconstructed using other channels.
  4. Section 3.3. The text indicates that iTransformer variate embedding is used and the global token is concatenated in the channel dimension. Yet, in L210 the authors say that the global token is concatenated to the temporal dimension of each token, but no such dimension exists after creating iTransformer-style N×DN \times D variate embeddings.
  5. Section 4.1. I do not understand why NINE and ELEVEN are uppercase and bold, perhaps this is a mistake?
Questions for authors
  1. Have you evaluated ResNo vs iTransformer on other non-gaussian noise types? How do other non-iTransformer-style models deal with noise?
  2. Was ResNo hyperparameter tuned in the same capacity as baselines for the evaluation on the 9 datasets?
  3. Are iCoIn and ResNo the same architectures?
Reviewer_cgziOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ResNo, a multivariate time-series forecasting framework intended to improve robustness to channel noise. ResNo consists of three components: (1) CoIn, which replaces pairwise attention with shared contextual aggregation followed by channel-wise gating; (2) a learnable global token that aggregates and redistributes global information; and (3) Channel Decoupling Loss (CDL), which creates auxiliary training pairs by subtracting randomly selected channels from both the input and target. The paper evaluates the method on nine forecasting benchmarks and reports favorable accuracy and inference efficiency, particularly on high-dimensional datasets such as PEMS and Traffic.

Critical strengths

1.The paper studies a relevant practical problem: uncontrolled propagation of corrupted sensor channels in multivariate forecasting. 2.CoIn is structurally simple and avoids constructing an N×NN\times N channel-attention matrix. Its linear scaling with the number of channels is plausible and potentially useful for high-dimensional datasets. 3.The evaluation covers multiple datasets and includes component ablations, plug-in experiments with PatchTST and iTransformer, efficiency measurements, and five-seed results. 4.The reported improvements on PEMS and Traffic are potentially meaningful, and the proposed modules appear easy to integrate into existing forecasting architectures.

Critical weaknesses
  1. The central robustness mechanism is not sufficiently established CoIn still aggregates every channel into a shared context HH. A corrupted channel can receive a large aggregation weight or contribute a corrupted value and therefore contaminate this shared context. The subsequent gate is channel-specific, but it only rescales the already mixed context and does not identify which part of HH originated from the corrupted channel. Consequently, the claim that CoIn localizes or prevents cross-channel noise propagation is currently supported mainly by intuition and a empirical study rather than by a formal perturbation bound or a targeted mechanism experiment.
  2. Robustness evaluation is insufficient to support the robustness claims. The robustness experiment corrupts only the last channel with Gaussian noise during training. It is unclear whether validation and test inputs are corrupted in the same way, whether a separate model is trained for each noise level, and whether the experiment measures training-time adaptation or robustness to unseen test-time corruption. Results involving randomly selected channels, multiple simultaneously corrupted channels, impulse noise, bias or drift, missing blocks, and train-test corruption mismatch would be necessary to support the broader claims about real-world channel noise. 3.Baseline comparison omits closely related recent methods with analogous design paradigms. All methods in the main comparison table are from 2024 or earlier. Relevant peer-reviewed baselines are missing, including SOFTS/STAR [1], ChannelTokenFormer [2], and CPiRi [3]. The omission of SOFTS is particularly notable because the paper explicitly states that its global-token redistribution follows STAR. The current comparison therefore provides limited support for the claimed state-of-the-art performance and makes it difficult to assess the contribution relative to recent cross-channel forecasting methods.

[1] Lu Han, Xu-Yang Chen, Han-Jia Ye, and De-Chuan Zhan. “SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion.” Advances in Neural Information Processing Systems 37 (NeurIPS 2024). https://proceedings.neurips.cc/paper_files/paper/2024/hash/754612bde73a8b65ad8743f1f6d8ddf6-Abstract-Conference.html [2] Jinkwan Jang, Hyungjin Park, Jinmyeong Choi, and Taesup Kim. “Towards Robust Real-World Multivariate Time Series Forecasting: A Unified Framework for Dependency, Asynchrony, and Missingness.” International Conference on Learning Representations (ICLR 2026). https://proceedings.iclr.cc/paper_files/paper/2026/hash/f924dab7a06d0463fe238904678acaee-Abstract-Conference.html [3] Jiyuan Xu, Wenyu Zhang, Xin Jing, Jiahao Nie, Shuai Chen, and Shuai Zhang. “CPiRi: Channel Permutation-Invariant Relational Interaction for Multivariate Time Series Forecasting.” International Conference on Learning Representations (ICLR 2026). https://proceedings.iclr.cc/paper_files/paper/2026/hash/ebba182cb97864368fdb6ae00773a5e4-Abstract-Conference.html 4. CDL may be better interpreted as data augmentation than as an explicit decoupling loss CDL constructs an auxiliary forecasting objective using channel-difference inputs and targets, and can also be viewed as a form of structured data augmentation. The implementation requires an additional forward pass for the auxiliary view, while the efficiency analysis focuses mainly on inference. Reporting the corresponding training overhead and comparing with related augmentation or regularization baselines, such as channel dropout, channel permutation, or random channel differencing, would help clarify whether the gains arise specifically from the proposed decoupling design. 5. The effective-weight analysis is not technically valid for the nonlinear model Appendix B.2 treats the trained ResNo model as f(x)=Ax+bf(x)=Ax+b and extracts a global “weight matrix” using zero and basis-vector inputs. This identity is valid for a linear model, but ResNo contains input-dependent normalization, softmax aggregation, sigmoid gates, GELU nonlinearities, and multiple stacked blocks. The extracted matrix is therefore an input-specific finite-difference probe rather than a global model weight. Its maximum eigenvalue and column-wise entropy cannot directly establish that CDL increases the model's representational capacity or reorganizes global channel coupling. A local Jacobian evaluated over representative real inputs, together with aggregate sensitivity statistics, would provide a more defensible analysis.

Questions for authors

1.Can the authors derive a perturbation or Lipschitz-style bound on how noise in one input channel affects other output channels under CoIn, and compare this bound with standard attention? 2.Are the training, validation, and test inputs all corrupted in Figure 5? Is a new model trained for every value of β\beta, and is the test corruption seen during training? 3.How does ResNo compare with SOFTS/STAR, an efficient-attention replacement, and representative peer-reviewed 2025–2026 forecasting models under the same data splits and tuning budget? 4.Does CDL outperform channel dropout, channel permutation, Mixup-style augmentation, and random channel differencing when the number of forward passes or total training FLOPs is matched? 5.Can the authors replace the global linear “weight matrix” interpretation with a local Jacobian or another mathematically valid input-dependent sensitivity analysis?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.