acceptodds
Under review as a conference paper at ICLR 2027

MixRoute: Mixed Parameterization with Adaptive Information Routing for Time Series Forecasting with Exogenous Variables

Abstract

Time series forecasting with exogenous variables (TSF-X) predicts an endogenous target series from its own history and from exogenous variables whose future values may be available in advance. The task therefore couples temporal dependence within the endogenous series with cross-variable dependence on the exogenous variables. Decoupled methods model these dependencies in separate modules and rely on their combination to capture correlations that span time and variables jointly. Joint methods instead model both dependencies in one token space, but parameter sharing and information routing remain important design considerations. Homogeneous parameterization applies the same transformations to endogenous patches and exogenous-variable tokens despite their different semantic roles. A single attention distribution per head couples temporal modeling with exogenous information acquisition, without explicit specialization for either role. To address these issues, we propose MixRoute, a unified Transformer framework that makes representation and information flow type-aware for endogenous patches, exogenous variables, and learnable global tokens. Under Mixed Parameterization, endogenous patches share one set of parameters, whereas each exogenous and global token has a private set. Adaptive Information Routing (AIR) uses separate queries for an intrinsic stream over the endogenous history and a cross-type stream over the exogenous and global tokens. It then combines the stream outputs with a gate computed from both streams. On 12 real-world datasets in the with-future setting, MixRoute reduces MSE by 8.0% on average relative to the strongest baseline evaluated. When future exogenous values are unavailable, MixRoute needs no architectural change and still ranks first overall.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, exogenous variables, heterogeneity

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_VWYyOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

MixRoute jointly processes endogenous patches, exogenous-variable tokens, and learnable global tokens. Endogenous patches share parameters, while each exogenous and global token has private attention and feed-forward parameters. Adaptive Information Routing (AIR) uses separate endogenous queries and separately normalized attention for an intrinsic history stream and a cross-type stream over exogenous and global tokens. An input-dependent gate scales the cross-type contribution. Global-token readout and an endogenous shortcut produce the forecast.

On twelve datasets, the paper reports an average per-dataset relative MSE reduction of 8.0% over DAG when future covariates are supplied, and 2.3% when they are withheld. The first setting achieves the best horizon-averaged MSE on all twelve datasets; the second does so on ten, with ties credited by the reported convention (Tables 1-2 and Section 3.2.2, pp. 6-8). These statements concern aggregate results, not a win at every individual horizon.

Critical strengths
  1. Clear architectural motivation. The sharing scheme addresses heterogeneous token roles, while AIR separates within-stream selection from stream combination. The formulation and main information paths are clearly described.
  2. Substantial controls for capacity and parameter sharing. Table 3 provides component ablations. Appendix C.6 scales the fully shared variant up to four times the reference parameter count, and Appendix C.7 includes parameter-matched exogenous-sharing and shared-query controls. It would be inaccurate to claim that the paper lacks controls for parameter count or separate queries (pp. 8 and 22-23).
  3. Evaluation under several input conditions. The paper studies missing covariates, unequal look-back lengths, frequency mismatch, delays, and absence of future covariates. It specifies retraining under the missing-value corruption, so those results should be interpreted accordingly (pp. 8-9).
  4. Transparent experimental disclosures. The authors identify inherited baseline results, future-input provenance, and limitations of high-dimensional scaling. The evaluation protocol specifies chronological splits and training-only standardization (pp. 13-14).
Critical weaknesses
  1. Main cross-model comparisons do not use matched search budgets. Tables 1-2 reuse baseline results from DAG. The manuscript follows the same benchmark framework and data protocol, which supports comparability, but this does not establish equivalent tuning effort. The main tables also lack run-to-run uncertainty. This matters particularly for the smaller advantage without future covariates.
  2. Deployment claims depend on the provenance of future covariates. Appendix A.1 distinguishes published electricity-market forecasts from recorded future generation and weather reanalysis. The latter provide idealized future information, rather than operational forecast errors. All compared methods are stated to receive the same variable availability, so this is an external-validity concern, not evidence of an unfair input advantage or demonstrated train-test leakage. Forecast issuance times should also be specified for each prediction origin and horizon.
  3. Private parameters limit high-dimensional scalability. Figure 11 reports 1,315.28 million parameters for an uncompressed Electricity stress-test configuration with 320 exogenous variables and hidden dimension 512. This number must not be presented as the size of every main-benchmark model: the main twelve datasets use only two to six exogenous variables. The proposed token compression substantially reduces size, but gains vary across settings (Appendix C.9, p. 25).
  4. The strongest architectural novelty lies in the combination and routing. The manuscript explicitly credits OneTrans for the parameter-sharing principle (p. 2). Table 3 supports an AIR contribution with the global-token head retained, so the larger effect of removing global tokens does not invalidate AIR. The remaining question is when AIR contributes sufficiently to justify its added structure, particularly beyond the reported domains.
Questions for authors
  1. Can DAG and GCGNet be rerun with comparable search budgets and multiple seeds to assess whether the reported margins are reliable?
  2. How do the results change when recorded or reanalysis-based future values are replaced with forecasts available at each prediction origin? Please specify forecast vintages and availability across the full horizon.
  3. Beyond the existing parameter-matched sharing controls, how does MixRoute compare with a shared backbone using variable embeddings or low-rank variable-specific adapters?
  4. Which dataset characteristics predict AIR's incremental benefit when global tokens remain in the model? Can this be related to covariate usefulness or changing cross-variable dependence?
  5. Can parameter count, latency, and memory be reported for the main low-dimensional tasks as well as the supplementary high-dimensional settings?
Reviewer_UmzgOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes MixRoute, a unified Transformer framework for time series forecasting with exogenous variables. The framework introduces Mixed Parameterization, which shares parameters across endogenous patches while assigning independent parameters to each exogenous and global token. It also incorporates Adaptive Information Routing (AIR), which uses separate attention streams to capture temporal dependencies within the endogenous series and interactions with exogenous and global tokens.

Critical strengths
  1. The overall framework is clearly presented and easy to follow, making the proposed methodology straightforward to understand.
  2. The experimental results demonstrate the effectiveness of the proposed approach across multiple real-world datasets.
Critical weaknesses
  1. The overall novelty of the proposed framework appears somewhat limited. AIR combines relatively established techniques, including separate attention streams and adaptive gating.
  2. Mixed Parameterization assigns independent QKV projection matrices and FFN parameters to each exogenous and global token. Although the ablation studies demonstrate performance improvements over shared-parameter variants, it remains unclear whether these gains arise from the proposed parameterization strategy or simply from increased model capacity. Could the authors report the parameter counts and computational costs of the different variants and include parameter-matched baselines to demonstrate that the improvements are attributable to the proposed design rather than additional parameters?
  3. The motivation for Mixed Parameterization is understandable, but the necessity of assigning independent parameters to every exogenous and global token is not sufficiently justified.
  4. Although Table 3 provides extensive ablation studies of the individual components of AIR, the experiments do not sufficiently demonstrate its advantages over existing attention mechanisms. In particular, the paper argues that conventional joint attention suffers from entangled temporal and cross-variable modeling, but this claim is not directly validated. It would be helpful to compare AIR with standard multi-head self-attention and conventional self-attention plus cross-attention within the same MixRoute framework, while keeping the other components unchanged. Such comparisons would better clarify whether the improvements come from the proposed information routing mechanism rather than simply separating attention streams.
Questions for authors
  1. The overall novelty of the proposed framework appears somewhat limited. AIR combines relatively established techniques, including separate attention streams and adaptive gating.
  2. Mixed Parameterization assigns independent QKV projection matrices and FFN parameters to each exogenous and global token. Although the ablation studies demonstrate performance improvements over shared-parameter variants, it remains unclear whether these gains arise from the proposed parameterization strategy or simply from increased model capacity. Could the authors report the parameter counts and computational costs of the different variants and include parameter-matched baselines to demonstrate that the improvements are attributable to the proposed design rather than additional parameters?
  3. The motivation for Mixed Parameterization is understandable, but the necessity of assigning independent parameters to every exogenous and global token is not sufficiently justified.
  4. Although Table 3 provides extensive ablation studies of the individual components of AIR, the experiments do not sufficiently demonstrate its advantages over existing attention mechanisms. In particular, the paper argues that conventional joint attention suffers from entangled temporal and cross-variable modeling, but this claim is not directly validated. It would be helpful to compare AIR with standard multi-head self-attention and conventional self-attention plus cross-attention within the same MixRoute framework, while keeping the other components unchanged. Such comparisons would better clarify whether the improvements come from the proposed information routing mechanism rather than simply separating attention streams.
Reviewer_FgXpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MixRoute, a Transformer-based framework for time series forecasting with exogenous variables. The authors argue that existing joint modeling methods do not sufficiently distinguish between endogenous temporal patches and exogenous variable tokens in either parameter sharing or information flow.

MixRoute introduces two main mechanisms. Mixed Parameterization shares parameters across endogenous patches while assigning private parameters to individual exogenous and global tokens. Adaptive Information Routing (AIR) separates intrinsic temporal modeling from cross-type information acquisition, using separate attention streams and a learned gate to combine their outputs. Global tokens are also used for information aggregation and prediction.

Experiments on real-world datasets show consistent improvements over existing methods. The paper also includes extensive ablation studies and additional evaluations under different exogenous data conditions.

Critical strengths
  1. Clear motivation and a well-designed framework. The paper identifies two specific issues in joint modeling of endogenous and exogenous variables: homogeneous parameter sharing and entangled attention. The proposed mechanisms directly address these issues. I find the overall design reasonable and well motivated.
  2. Strong empirical results with careful ablation studies.MixRoute achieves consistent improvements across multiple datasets. The authors also provide fairly thorough ablations, including capacity-controlled comparisons, parameter-sharing strategies, and separate attention queries. These experiments offer useful evidence for the proposed design. The additional evaluations under different exogenous data conditions are also appreciated.
Critical weaknesses
  1. The novelty of the proposed mechanisms could be better established. Several important ideas have been explored in previous work. TimeXer already uses endogenous patches, exogenous tokens, and global tokens with separate attention mechanisms, while OneTrans adopts a similar parameter-sharing principle for heterogeneous tokens. MixRoute combines these ideas in a different way, but I am not fully convinced that it introduces a substantially new modeling mechanism. The ablation studies support the effectiveness of AIR, which I appreciate. However, since global tokens already provide an indirect path for exogenous information, it would be useful to better explain when the additional cross-type stream offers a clear advantage.

  2. Some aspects of the evaluation limit the strength of the conclusions. The results are encouraging, but several datasets use future exogenous observations or reanalysis data rather than forecasts available in advance. Although this follows an established benchmark protocol, the practical effectiveness of MixRoute under realistic future-covariate uncertainty remains less clear. Also, the main baseline results are taken from the DAG paper rather than independently reproduced. The evaluation follows the same benchmark settings, but a more controlled comparison with the closest methods would further strengthen the empirical claims.

Questions for authors
  1. What is the main advantage of AIR over existing information fusion mechanisms? TimeXer already separates temporal modeling and exogenous information acquisition. Could the authors clarify what additional capability AIR provides, beyond the existing global-token-based interaction?

  2. When is the direct cross-type stream particularly useful? The ablation results suggest that global tokens play a major role in the final prediction. Are there specific data characteristics or types of exogenous dependencies where direct cross-type routing provides a larger benefit?

  3. How robust is MixRoute to errors in future exogenous inputs? Some datasets use recorded future values or weather reanalysis data. Could the authors evaluate the model with imperfect future covariate forecasts, or discuss how this might affect the reported advantages?

  4. How well does the method scale to larger numbers of exogenous variables? The token-compression experiments in Appendix C.9 are useful, but the performance gains are not consistent across all settings. Could the authors further clarify the trade-off between parameter efficiency and forecasting accuracy, particularly when the exogenous variable set becomes large or changes over time?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.