acceptodds
Withdrawn from ICLR 2027

AltTS: Decoupling Autoregression and Cross-Variable Dependency via Alternating Optimization for Multivariate Time Series Forecasting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multivariate time series forecasting requires learning both within-variable temporal dependencies and cross-variable interactions. When these components are optimized jointly, their predictions enter a shared residual, coupling their learning signals. We introduce ALTTS, a dual-path forecasting framework that explicitly separates temporal and cross-variable modeling and coordinates their adaptation through staged alternating optimization. The framework combines a linear temporal predictor with a masked cross-variable attention module. Its training procedure first establishes a temporal predictor, fits a cross-variable residual correction, and then alternates between the branches with block relaxation. Our analysis characterizes residual-induced gradient coupling, and local learning-signal diagnostics illustrate how this interaction can affect temporal updates. Experiments on seven forecasting benchmarks demonstrate competitive accuracy using standard architectural components. These results highlight branch coordination as an effective design dimension for multivariate forecasting. Code available at: anonymous github.

Primary area:
learning on time series and dynamical systems
Keywords:
multivariate time series, time series forecasting, alternating optimization, autoregression, cross-variable dependency

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MAMuOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AltTS, a dual-path framework for multivariate time series forecasting that separately models within-variable temporal dependencies and cross-variable dependencies through staged alternating optimization.

Critical strengths

S1,The proposed method is relatively simple, with a clear framework that is easy to understand and implement.

S2,The paper investigates multivariate time series forecasting from an optimization perspective rather than focusing on architectural design, providing an interesting research direction for coordinating different forecasting components.

Critical weaknesses

W1. Limited methodological novelty. The proposed method mainly combines existing components, including a linear predictor, cross-variable attention, and alternating optimization. The overall approach appears to be an incremental improvement over existing techniques.

W2. Insufficient comparisons with recent strong baselines. The experiments lack comparisons with recent competitive time series forecasting methods, such as SOFTS, DUET, FACT, GTR, and VPNet. Therefore, the current results are insufficient to establish the competitiveness of the proposed method.

W3. Insufficient justification of the core motivation. Gradient coupling does not necessarily imply harmful optimization interference, as both branches optimize the same forecasting objective. The paper does not provide sufficient evidence that the performance improvements of alternating optimization result from mitigating harmful gradient interference.

W4. Insufficient ablation studies. The individual contributions of key components, including staged initialization, alternating optimization, and block relaxation, are not adequately evaluated, making it difficult to identify the main sources of performance improvements.

Questions for authors

Q1. What is the core technical novelty of AltTS compared with existing model architectures and alternating optimization methods?

Q2. Could the authors provide comparisons with recent competitive methods, such as SOFTS, DUET, TimeFilter, FACT, GTR, and VPNet, under consistent experimental settings?

Q3. How can the authors demonstrate that gradient coupling indeed causes harmful optimization interference? Could the authors further analyze gradient behavior under different training strategies and its relationship with forecasting performance?

Q4. Could the authors provide additional ablation studies to evaluate the individual contributions of staged initialization, alternating optimization, and block relaxation, while ensuring a fair computational budget compared with joint training?

Reviewer_4oX6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ALTTS, a dual-path forecasting framework that explicitly separates temporal and cross-variable modeling and coordinates their adaptation through staged alternating optimization. Within-variable temporal dependencies and cross-variable interactions are two critical aspects of multivariate time series forecasting tasks; however, when optimized jointly, their predictions share a common residual, coupling their learning signals and causing the model to mutually interfere when capturing these two aspects. To address this, this paper combines a linear temporal predictor with a masked cross-variable attention module and alternates between the branches using block relaxation. Experiments on seven forecasting benchmarks demonstrate the competitive accuracy of ALTTS and highlight branch coordination as an effective design dimension for multivariate forecasting.

Critical strengths
  1. This paper aims to address the challenge of jointly optimizing within-variable temporal and cross-variable predictors by focusing on the gradient coupling between them. This provides a relatively novel perspective on multivariate time series forecasting.
  2. The competitive results across multiple datasets demonstrate the effectiveness of ALTTS and provide experimental evidence of the improvements achieved by alternating optimization compared to the joint-training baseline.
Critical weaknesses
  1. The motivation of this paper is to address the gradient coupling arising from the joint optimization of within-variable temporal dependencies and cross-variable dependencies; however, in Section 3.3, when training the AR path, the prediction results from the CR path are also included in the loss. Therefore, the authors do not actually eliminate gradient coupling. Please provide further clarification on the actual purpose of this step.

  2. In ALTTS, the AR path uses a simple linear prediction model from RLinear, while the CR path uses a simple attention mechanism to model variable relationships from iTransformer. Since both paths are based on existing works, the contribution and novelty of the proposed framework are highly limited. Furthermore, the authors have not compared the performance of the AR path and the CR path under different approaches. For example, how would the results change if the AR path used a more complex model, such as PatchTST, or if the CR path employed a simple linear mapping? Would the conclusions remain consistent across different architectures?

  3. The paper does not provide detailed hyperparameter configurations to enhance reproducibility, such as αA\alpha_A, αC\alpha_C, and K cycles in the "Block-relaxed alternating updates" section. Furthermore, the sensitivity of model performance to these hyperparameters is also not investigated.

  4. The paper does not sufficiently isolate the effects of staged initialization and block-relaxed alternating updates through independent ablation experiments. Consequently, it remains unclear how much of the performance improvement comes from staged initialization rather than the subsequent alternating optimization. Additionally, the paper does not demonstrate whether block-relaxed alternating updates section is effective and quantify what extent they contribute to performance.

Questions for authors
  1. The paper first trains the AR path and then uses the CR path to fit the residual error; however, since the relationships between variables in the ETT datasets are very weak, this initialization may be more suitable for these datasets. For other high-dimensional multivariate datasets, such as the Traffic and PEMS datasets, would it be a better choice to first train the CR path and then use the AR path to fit the residual error? Could the authors provide an explanation or experimental comparison of these two initialization strategies?

  2. The AR path primarily models temporal dependencies, while the CR path is designed to capture cross-variable dependencies. However, the CR path first maps variable tokens to a high-dimensional embedding space and preserves the variable's own temporal information. Does this introduce an overlap with the autoregression in the AR path, potentially leading to insufficient functional decoupling and redundancy between the two branches?

  3. The experiments use a fixed input length of 512 for all baselines, while an input length of 96 is also commonly adopted in many multivariate forecasting baselines, such as OLinear and iTransformer. To ensure fair comparisons, the authors should additionally report performance with an input length of 96, as the authors' own reproductions of these baselines may not accurately reflect their actual performance and could potentially underestimate their forecasting capabilities.

Reviewer_GbkpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AltTS combines a linear, channel-independent temporal predictor with a masked cross-variable Transformer correction. Training first fits the temporal branch, then fits the correction to its residuals, and finally alternates parameter-block updates with relaxation. The paper derives how a shared prediction residual changes the temporal gradient and presents local gradient interventions. Seven forecasting benchmarks compare the complete training procedure with joint training and existing forecasters; further analyses remove branches at inference and vary the alternating schedule.

Critical strengths
  • The paper studies an important problem in time series forecasting.
  • The limitations of the optimization analysis are explicitly acknowledged.
Critical weaknesses
  • The central comparison does not isolate alternation. Table 2 compares a joint-training control with a procedure that also introduces staged AR pretraining, residual fitting of CR, asymmetric training passes, branch-specific optimization, relaxation, and validation selection. It is unclear whether initialization, update budgets, learning-rate tuning, and checkpoint selection are matched. This is the principal evidential weakness: without that information, the improvement cannot be attributed specifically to alternating optimization.

  • Schedule comparisons may confound scheduling with training budget. Table 4 changes block lengths and therefore can change the number of training passes. Without a stated fixed update or compute budget, it cannot isolate schedule quality. Report the total updates, cycle counts, and stopping rule for each setting; if budgets differ, interpret the table as a comparison of complete training recipes rather than evidence for a superior schedule.

  • The benefit of relaxation is not separated from ordinary alternation. The study does not include a comparison isolating block relaxation from the rest of the training procedure, and selected relaxation coefficients and other essential optimization settings are incompletely reported. The authors should provide the existing configuration records and distinguish evidence for the whole recipe from evidence for this particular ingredient.

  • The gradient identity does not establish harmful interference. Equations 9–13 compare the combined predictor's gradient with the gradient of a different, AR-only objective. A difference is expected for additive squared-error predictors; its existence does not demonstrate that the coupling is harmful. Section 3.3 correctly acknowledges that alternation leaves the instantaneous active partial derivative unchanged at an identical state. The proposed mechanism therefore remains a trajectory-level hypothesis and should be presented that way.

  • The proposed variance mechanism is not linked to the actual training runs. The sufficient variance condition is not shown to hold in the reported runs, nor is the alternating procedure shown to reduce that quantity. The selected ETTh illustration and one-step interventions offer limited support for the multi-pass Adam procedure. The authors should identify existing trajectory-level evidence or substantially qualify the explanatory claim.

  • Inference-time branch removal is not an independently trained control. Table 3 removes branches from the trained combined predictor. This does not show how separately trained AR-only or CR-only models would perform, so it cannot establish the benefit of the proposed coordination relative to those alternatives. The table should be described as dependence of the fitted model on its branches, rather than a substitute for matched training ablations.

Questions for authors
  1. Does the Table 2 joint baseline start from the exact same staged AR and residual-CR checkpoints? Please provide its branch learning rates, regularization, update counts, early stopping, and checkpoint-selection protocol alongside those of AltTS.

  2. In Table 4, are total branch updates or total compute held constant across schedules? What are the cycle count and the selected CR relaxation coefficient in the main results?

  3. Is there an already completed comparison separating staged initialization, unequal branch-update budgets, ordinary alternation, and relaxation?

  4. Which existing diagnostic links the actual alternating Adam trajectory to reduced harmful coupling, beyond the elementary gradient identity and the selected local illustration?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.