acceptodds
Under review as a conference paper at ICLR 2027

DepShift: Measuring What Cross-Channel Modeling Buys, and What It Costs Under Dependency Shift

Abstract

A recent method proposes channel shuffling as a certificate that a multivariate forecaster does not over-rely on cross-channel structure. It certifies less than it is taken to. Shuffling is a channel permutation, a relabeling of a coupling that stays intact; the disturbance a deployed model faces is a rewiring, the coupling changing with channel identities fixed. A relabeling also permutes the per-channel marginals unless the channels it swaps share one law, whereas a change in the coupling's lag leaves every single-channel law exactly unchanged. We build DepShift, a generator on which the lag change is the pure cross-channel disturbance and a relabeling is not, and an excess-fragility contrast against a channel-independent control. Permutation-equivariance does not confer robustness to rewiring: the same model is more fragile to a rewiring than to a relabeling at 11 of 12 coupling configurations, null in the factor topology, and the flip holds for CPiRi's released code at 5 of 5, uncorrected by its own shuffling regulariser. Along a dose axis, and replicated on six fresh seeds, the equivariant model's response to a lag change grows far faster than its response to a relabeling. What permutation degradation it does show is its evaluation pipeline's, and the same normalisation switch leaves the fragility of all three identity-storing architectures we put through it in place. The two classes also separate out of sample at full channel count. To read a null one must know what was achievable, so we compute a closed-form ceiling for the cross-channel increment: against it the equivariant model pays a tenth to a quarter of the optimal predictor's price under a lag change and under a twentieth under relabeling. We report where this fails: the ceiling does not transfer to real data, 3 pre-registered attempts to attribute the real-data ordering to coupling movement failed, the real-data test is met on one of three datasets, and a ratio-metric artifact produced a retired hypothesis's predicted sign at p=2e-59 inside a protocol written by authors trying not to be fooled.

Primary area:
learning on time series and dynamical systems
Keywords:
multivariate time series forecasting, distribution shift, dependency shift, channel dependence, permutation equivariance, robustness evaluation, pre-registration, benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vrkeOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces controlled dependency shifts, a channel-independent fragility control, and Gaussian oracle baselines for multivariate forecasting. Experiments show that robustness to channel permutations does not ensure robustness to changing lag relationships.

Critical strengths

The lag-change generator preserves individual channel laws, enabling an interpretable test of dependency changes.

Channel-access ablations strengthen the central synthetic finding.

Critical weaknesses

The conceptual formulation largely follows from permutation equivariance. Its additional empirical significance remains limited by positive findings concentrated on directed lag chains and unsuccessful real data attribution.

Synthetic experiments use fixed training budgets without validation-based selection. Since checkpoint selection reverses several other reported findings, the optimization sensitivity of these comparisons remains unresolved.

Questions for authors

NA

Reviewer_K7KUOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper examines the predictive benefits and robustness costs of cross-channel modeling in multivariate time-series forecasting. It introduces Depshift, a data generator that produces multivariate time series with controlled inter-channel coupling and test-time shifts in these dependencies. The authors propose a metric that quantifies the increase in forecasting error under a dependency shift relative to a channel-independent baseline. They also derive a closed-form upper bound on cross-channel predictive gains for linear-Gaussian generators. Experiments show that robustness to channel permutations does not guarantee robustness to dependency shifts.

Critical strengths

The topic and idea should be of interests to a certain group of ICLR readers.

Critical weaknesses

The paper lacks explanatory figures, and Table 1 in the main text is difficult to interpret. Many points are repeatedly emphasized throughout the manuscript. The experimental section has not been properly integrated: earlier experiments and supplementary results are presented side by side.

Especailly, AI-generated revision history has also been included directly in the manuscript....

Questions for authors
  1. What actionable insights does this study offer into when cross-channel modeling is beneficial, and how can these insights inform model selection or design under dependency shifts?
  2. What theoretical or empirical evidence supports the design of the proposed excess-fragility metric?
Reviewer_L4tFOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies what channel-shuffling robustness does and does not reveal about multivariate forecasters. It distinguishes relabeling channels from changing the underlying cross-channel coupling, and introduces DepShift generators and an excess-fragility evaluation against a channel-independent control. The paper also computes exact cross-channel forecasting value for its linear-Gaussian constructions and studies whether trained models actually use the available information. Synthetic and real-data experiments examine the architectural conclusions and report both supporting and negative results.

Critical strengths
  1. The distinction between channel relabeling and a genuine dependency change is important. The paper provides a concrete reason why passing a permutation test should not be interpreted as a general certificate of robustness to changing relationships.

  2. The controlled generators, exact Gaussian forecasting risks, and channel-reliance measurements help distinguish a lack of useful cross-channel information from a model's failure to exploit it. This is a useful contribution to how multivariate forecasting methods are evaluated.

  3. The paper provides an unusually detailed account of its experimental scope and negative findings. The supplied implementation and logs support checking the principal synthetic measurements, rather than relying only on aggregated performance tables.

Critical weaknesses
  1. The main contributions lie in distinguishing channel relabeling from dependency shifts and developing an evaluation framework around this distinction. However, this conceptual distinction is relatively straightforward, and the proposed metrics and closed-form ceiling largely build on existing ideas. The paper does not sufficiently establish substantial methodological or theoretical advances beyond this combination.

  2. The proposed closed-form ceiling fails on real data, producing an invalid −368% result on Weather, and the authors consequently withdraw the corresponding real-data claim. Moreover, all three preregistered attempts to attribute real-data performance differences to dependency shifts fail, and the registered criterion is satisfied on only one of the three datasets. The evidence therefore remains predominantly synthetic, limiting the practical significance and generalizability of the conclusions.

  3. The paper fails to establish that dependency shifts are responsible for the observed architectural fragility, while some experiments cannot rule out general distribution shifts as an alternative explanation. Furthermore, the authors continue conducting attribution experiments after the preregistered stopping criterion has been triggered and disclose post-hoc choices favorable to their conclusions. These issues weaken the reliability of the experimental evidence.

  4. The appendix acknowledges that Autoformer and FEDformer were evaluated with an incorrect "label_len=0" setting, causing their decoders to receive the full lookback window rather than the intended 48-step label prefix. These models exhibit some of the largest excess-fragility results. Although the authors argue that this issue does not affect their main analysis, the deviation from the intended experimental settings raises concerns about the validity and interpretability of these comparisons.

  5. The paper is difficult to follow due to its convoluted writing and frequent digressions. In particular, the repeated pattern of presenting a result and immediately qualifying or undermining it with statements about what the authors “do not claim” disrupts the narrative and makes the main contributions and conclusions unnecessarily difficult to assess

Questions for authors
  1. What methodological advance and practical conclusions remain beyond the established relabeling-versus-dependency-shift distinction, given the limitations of the real-data diagnostic and attribution results?

  2. What evidence for dependency-specific architectural fragility remains when the preregistered stopping rules are respected and post-hoc choices are excluded?

  3. How does correcting the decoder label-prefix setting affect the Autoformer and FEDformer results and the architectural comparisons that use them?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.