acceptodds
Withdrawn from ICLR 2027

What Makes Cross-Variable Mixing Effective? Variable-Specific Value Interfaces for Multivariate Time-Series Forecasting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent multivariate time-series forecasting methods increasingly exploit cross-variable dependencies through mixing mechanisms that determine which variables communicate and how strongly. However, variables often differ in periodicity, phase, lag, and predictive role. Selecting a useful source therefore does not determine how its information should be expressed for different targets, yet existing mixers typically aggregate values in a shared latent space. We revisit cross-variable aggregation through variable-specific value interfaces. Our FrameMix operator equips variables with learnable orthogonal value interfaces for expressing outgoing information and interpreting aggregated information. Composing the source and receiver interfaces yields a distinct value transformation for every ordered variable pair without directly learning pairwise operators. We incorporate FrameMix into Frameformer, a patch-based architecture that mixes variables at each patch position before modeling temporal dependencies within each variable. Experiments show that variable-specific interfaces improve multiple routing mechanisms and remain effective with both learned and fixed frames, indicating that their gains cannot be explained by additional trainable capacity alone. More expressive pair-specific alternatives require substantially more parameters but provide no reliable accuracy gains. Frameformer achieves the lowest error on six of eight benchmarks at horizon 96 and is most effective on strongly coupled multivariate data, while results at horizons 192 to 720 on six benchmarks are mixed.

Primary area:
learning on time series and dynamical systems
Keywords:
multivariate time-series forecasting, cross-variable mixing, value interfaces, orthogonal frames

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_9LMxOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the problem of cross-variable dependency modeling in multivariate time-series forecasting. The authors argue that existing methods primarily focus on information routing between variables while overlooking the differences in how different variables express and receive information. To address this issue, the paper proposes FrameMix, which enables variable-specific information interactions by introducing learnable orthogonal transformations for each variable, and integrates it into a patch-based forecasting framework called Frameformer. Experiments on eight public datasets demonstrate that the proposed method achieves competitive forecasting performance on several datasets.

Critical strengths

S1: This paper focuses on the representational differences between variables during cross-variable information exchange and decomposes conventional cross-variable mixing into two components: information routing and variable-specific value transformations. This perspective provides meaningful insights into understanding the effectiveness of cross-variable dependency modeling.

S2: The authors conduct experimental analyses covering different routing mechanisms, orthogonal transformation strategies, fixed and learnable transformations, the respective roles of source and target variables, and model parameter complexity.

Critical weaknesses

W1: The paper mainly conducts comprehensive model comparisons with a prediction horizon of 96. Although Appendix F provides additional results for prediction horizons of 192, 336, and 720, only Frameformer's long-term forecasting results are reported on the PEMSD4 and PEMSD8 datasets, without corresponding results from other baseline models under the same prediction horizons. Therefore, the current experiments are insufficient to demonstrate that the proposed method consistently maintains its advantages across different prediction horizons. In particular, considering that cross-variable dependencies may change with the prediction horizon, comprehensive multi-horizon comparisons are essential for validating the effectiveness of the proposed method.

W2: Although Frameformer performs well on some datasets, its advantages are not consistent across all forecasting scenarios. For example, when the prediction horizon is 720, Frameformer achieves the lowest MSE on only two datasets, with noticeable performance gaps on the remaining datasets. Similarly, at a prediction horizon of 336, it achieves the best performance on only three datasets.

W3: The paper adopts a fixed learning rate of 0.001 and limits the maximum number of training epochs to 15 for all models. However, different models may have substantially different convergence rates and optimal hyperparameter configurations. The authors also explicitly state that the experiments follow a unified training protocol rather than performing independent hyperparameter tuning for each model. Therefore, the current results cannot sufficiently rule out the possibility that inadequate training or suboptimal hyperparameter settings of the baseline models affect the performance comparisons.

W4: Although the paper discusses recent cross-variable modeling methods such as FACT and xCPD in the related work section, these methods are not included in the main experimental comparisons. Considering that these studies also focus on fine-grained variable interactions or cross-variable dependency modeling, they are highly relevant to the research objective of this paper. The lack of direct comparisons makes it difficult to clearly establish the performance advantages and technical differences of FrameMix over recent cross-variable modeling methods.

W5: FrameMix applies orthogonal transformations to both source and receiver variables to enable information transformation between variable pairs. However, the experimental results in Figure 5 show that using only receiver-specific transformations already achieves most of the performance improvements and even outperforms the full model on the Weather and Electricity datasets. This suggests that the performance gains may primarily come from receiver-specific representations rather than the bidirectional composition of source and receiver transformations emphasized by the authors. Therefore, the necessity of the complete pairwise transformation design requires further justification.

W6: Frameformer modulates variable-specific orthogonal transformations across different patch positions to adapt to temporal variations in feature representations. However, Table 14 in Appendix G.3 shows that static transformations and patch-position-modulated transformations achieve nearly identical performance on most datasets, with a noticeable difference only on the Solar dataset. This indicates that the additional patch-position modulation mechanism does not provide consistent improvements in most cases. I hope the authors can further clarify why this design is necessary and explain its unique advantages over simpler static transformations.

Questions for authors

Q1: The authors mention that their experimental setup follows the protocol of CSformer, which in turn adopts the experimental settings of iTransformer. However, some results reported in Table 3 appear to differ substantially from those reported in the original iTransformer paper. I hope the authors can further clarify the reasons for these discrepancies and provide the complete source code, experimental scripts, and relevant hyperparameter configurations to improve the transparency and reproducibility of the experimental results.

Q2: The paper does not sufficiently cover recent advances in multivariate time-series forecasting published in 2026, with some relevant studies and corresponding baseline comparisons missing. I encourage the authors to include more representative recent works and conduct additional comparisons with these methods to provide a more comprehensive evaluation of the novelty and performance advantages of the proposed approach.

Reviewer_DYCjOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the cross-variable aggregation problem in multivariate time-series forecasting. It proposes the FrameMix operator, which assigns variables with adaptive orthogonal value interfaces. Afterwards, it develops FrameFormer, which integrates the proposed FrameMix operator for time-series forecasting. The effectiveness of the proposed method is validated on standard time-series benchmarks with fruitful in-depth analysis.

Critical strengths

The research problem is clearly formulated, while channel dependency has been identified as an important aspect to improve multivariate time-series forecasting. The experiments are equipped with in-depth analysis which contains fantastic insights. Authors also frankly introduce their use of AI tools in conducting experiments and drafting this paper.

Critical weaknesses
  • Theoretical analysis warrants further improvement. Specifically, it is essential to demonstrate why the proposed method is superior in certain cases and beneficial for forecasting task. While authors provide a proposition in p. 21, it seems a linear-algebra characterization, but does not concern forecasting error and model performance.

  • Reproducibility matters in evaluating the technical quality of submissions. In the current version, code implementation is not available, and a thorough introduction to the process / protocol to fine-tune the hyperparameters of the proposed method and baselines is not available. These factors reduce the reproducibility of this manuscript.

  • Experiments.

    • The baselines selected are somewhat out-of-dated. Most of baselines are developed prior to 2025, and only one baseline is developed in 2025. Comparison with more recent methods would be essential to posit this work better and highlight its significance.
    • The evaluation metric is limited. Most experiments only report MSE, which is pervasive but insufficient to depict forecasting performance. Alternatives such as MAPE and R2 should be considered to be involved to improve the comprehensiveness of metrics.
    • Authors report std in Table 3, but it is not clear whether the improvement is statistically significant. Authors should select and perform proper significant test in all significant experiments, which will help to present the superiority of the proposed method.
  • Presentation.

    • Some references are not complete, e.g., Spectral temporal graph neural network for multivariate time-series forecasting. The style and format of them should be unified (e.g., url).
Questions for authors

Kindly see the weakness section above.

Reviewer_Yc4jOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FrameMix, the key idea is to introduce a trainable matrix U_i for variable i when conducting patch-level attention. The variable specific transformation help to improve cross-variables prediction performance. The paper then test this idea against several existing models. Also test trainable transformation matrix vs. other random/affine matrix.

Critical strengths
  1. the presentation of the paper is good. easy to follow and understand.
  2. The experimental analysis in terms of the transformation itself is comprehensive. Through the evaluation of various routing mechanisms, fixed/random transformations, learnable transformations, and ablation studies, the impact of variable-specific interfaces on predictive performance is thoroughly analyzed.
  3. The results are significant on certain datasets, such as PEMS traffic datasets
Critical weaknesses
  1. The idea is simple and not compared against many similar ideas. Though the paper propose to have source and receiver transformation U_j and U_i, it tends out receiver only U_i is enough to capture most of the gains. Also, fixed random transformation is also helpful. Hence, from the current results, it is hard to justify the value of the FrameMix. In my opinion, the improvement mostly comes from adding a variable-specific adapter (random matrix is like assigning a unique identifier to a variable). So i am not sure FrameMix is really the key solution here.

  2. It is unclear what is the main contribution compares with other variable adapter related studies. For example, what if I just add an additional embedding to each variables when doing the attention. Does the orthogonal structure really needed? The paper should compare their results with related approaches (instead of comparing with general single-variable models), such as FACT and xCPD. In this case, the experiement is not enough.

Questions for authors
  1. Fixed random transformations and non-orthogonal affine transformations can also yield significant performance gains. Could the authors clarify the unique advantages of the proposed orthogonal "FrameMix" compared to general variable-specific transformations? Furthermore, what are the core differences and innovations of this approach in relation to existing work?
  2. Experiments indicate that transformation matrices with fixed random initialization can also improve predictive performance across multiple datasets. Does this suggest that the performance gains stem primarily from variable-specific representations and the co-adaptation between these transformations and the subsequent shared modules, rather than from the learned orthogonal transformations themselves? Could the authors further disentangle the contributions of these different factors?
  3. Most experiments rely on only three random seeds, and the performance differences between some methods are marginal. Could the authors provide more robust statistical tests or results based on a larger number of random seeds to verify whether the performance improvements over existing baselines are statistically significant and stable?
  4. The paper primarily highlights the advantages of "Frameformer" for a prediction horizon of 96; however, it lacks comparisons with other baselines for longer prediction horizons on the PEMS dataset. Could the authors include additional experiments to validate the effectiveness and scope of the proposed method for long-term forecasting tasks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.