acceptodds
Under review as a conference paper at ICLR 2027

CausalExoFormer: End-to-End Lag-Aware Causal Structure Learning for Exogenous Time-Series Forecasting

Abstract

Exogenous time-series forecasting requires identifying relevant external variables, capturing their delayed effects, and modeling interactions among them. These dependencies are often learned implicitly, leaving the structure of external information use difficult to inspect and control. We propose CausalExoFormer, an end-to-end Transformer that jointly learns variable relevance, temporal allocation, and directed cross-channel structure for forecasting. A factorized source–lag gate selects historical observations, a directed graph learned with acyclicity regularization models interactions among external variables, and causal-gated cross-attention integrates the resulting representations with the target history. The learned structure directly controls information flow: closing a source gate removes both its direct and graph-mediated contributions. For multivariate forecasting, a shared graph supports target-specific predictions through exact self-exclusion, preserving the separation between each target’s own history and its external inputs. Experiments on six standard benchmarks and six lake dissolved-oxygen forecasting tasks demonstrate the effectiveness of this approach. CausalExoFormer achieves the lowest horizon-averaged MAE on all six standard benchmarks and the lowest horizon-averaged MSE on four, among the evaluated methods, while consistently improving over TimeXer on the six lake tasks. Ablation studies establish the contribution of learned source–lag allocation, and environmental case studies reveal the variables and historical offsets used for prediction. Code is available at https://anonymous.4open.science/r/CausalExoFormer-C42C.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Time Series Forecasting, Exogenous Variables, Causal Structure Learning, Transformers, Lag-Aware Modeling, Directed Acyclic Graphs, Multivariate Forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_XGvQOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes CausalExoFormer, which explicitly learns variable relevance, historical lags, and interactions among exogenous variables for time-series forecasting. Building on TimeXer, it integrates a source–lag gate, a directed graph with NOTEARS-based acyclicity regularization, and null-augmented cross-attention, together with a shared graph and target-specific self-exclusion for multivariate forecasting. Proposition 1 establishes computational information-flow exclusion under gate closure. Across six standard benchmarks and six lake dissolved-oxygen datasets, the model achieves the best horizon-averaged MAE on all six standard benchmarks and MSE on four. The paper also reports ablations, synthetic source-and-lag recovery, and a real-data interpretation case study.

Critical strengths
  1. Operational control of information flow. The gate acts on both direct and graph-mediated pathways, and Proposition 1 shows that closing a gate excludes the corresponding observation from the external forecasting pathway.
  2. Explicit separation of source relevance and lag allocation. The factorized source–lag gate makes these quantities separately inspectable, and the synthetic experiments report stronger source and lag recovery than PCMCI+ and DYNOTEARS.
Critical weaknesses
  1. (Readability) Although the paper is generally well organized, its readability is hindered by cumbersome notation and inconsistencies between the architecture figure and the equations. In Sections 2–3, the numerous indices and mappings, including SS, CC, σ(r)\sigma(r), r(c)r(c), and Πm\Pi_m, make the central mechanism difficult to follow. The training objective in Figure 1 contains LconsistL_{\mathrm{consist}} and λ2,λ3\lambda_2,\lambda_3, which are absent from Equation (11), while the cross-attention diagram omits the null alternative in Equation (9). The descriptions of zero diagonal and self-exclusion may also be conflated, and the section reference for the training schedule is incorrect. These issues obscure the correspondence between the illustrated architecture and its actual computation.

  2. (Reproducibility) The computational properties of the proposed method are described mathematically, but the experimental evidence does not sufficiently support its central claims. The standard-benchmark and lake forecasting experiments do not report numbers of repeated runs or variability across seeds, while many improvements over strong baselines are on the order of 1–2%. In particular, the reported 8.1% average MSE improvement over TimeXer on Weather relies heavily on the gain at H=720H=720; excluding that horizon reduces the improvement to approximately 1.8%.

In the ablations, removing the graph or DAG penalty increases average MSE by only about 1.4% each, and some datasets improve when these components are removed. These ablations are restricted to H=96H=96, limiting evidence for a consistent contribution across forecasting horizons. Furthermore, although the synthetic data have a known ground-truth directed graph, Table 3 evaluates only source and lag recovery, not recovery of the learned graph BB. Finally, the standard-benchmark baseline results are taken from earlier publications, while the execution settings for several lake baselines and the precise lake station identifiers and observation periods are not sufficiently documented. The procedure for selecting the default hyperparameters used in the sensitivity analyses also remains unclear, limiting the assessment of reproducibility and comparative reliability.

  1. Weak points
  • Missing validation of directed-graph recovery. Even with a known synthetic ground-truth graph, the paper does not report how accurately the learned BB recovers it. Computational pathway exclusion does not establish causal correctness.
  • Insufficient statistical support for performance gains. Many improvements are small, but repeated-seed means and variability are not provided for the forecasting experiments; some gains are concentrated in particular horizons.
  • Limited contribution of the graph and DAG penalty. Their removal causes only small degradations on average and sometimes improves performance, while the ablations cover only H=96H=96.
  • Incomplete reproducibility details. The comparisons reuse published baseline numbers, and the lake baseline settings, precise station and observation-period information, and hyperparameter selection protocol are insufficiently specified.
  • Limited structural interpretation. The synthetic experiment uses a linear SCM, and the real-world interpretation focuses on a few variables from Erie2, without a systematic assessment of structural stability or domain validity.
Questions for authors
  1. Graph Recovery: Given that the synthetic data include a ground-truth directed graph, why are edge precision, recall, structural Hamming distance, or comparable recovery metrics for the learned BB not reported? Can these results be provided?
  2. Contribution of Graph Learning: Removing the graph or DAG penalty changes average MSE by only about 1.4%, and some datasets improve. What additional evidence demonstrates a substantive benefit beyond the source–lag gate? Does this benefit persist at other horizons?
  3. Statistical Reliability: How many random seeds were used in the standard and lake forecasting experiments? Can means and standard deviations from repeated runs be provided, particularly against closely performing baselines such as CrossLinear?
  4. Experimental Protocol: What were the training settings for the lake baselines other than CrossLinear? How were the default hyperparameters in Tables 10–12 selected independently of the test sets?
  5. Causal Interpretation: How do the authors distinguish the computational exclusion established by Proposition 1 from recovery of causal structure? What assumptions would justify interpreting the learned directed graph causally?
Reviewer_F8GVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

From what I understand, on top of the conventional time series signals, the approach will store certain external, time dependent, variables in a memory bank, including a series of structural constraints stored else where. The proposed method, CausalExoFormer, combines these constraint sources with features from the actual time series, creating structural and external data constraints. These quanties are continuously updated as the signal rolls out, allowing for enhanced context into predictions of the actual variables.

Critical strengths
  1. The combination of structural constraints in terms of a DAG and the exogenous variables seems novel, though I struggle to grasp the novel contribution of this paper, and will defer to the authors for future discussion.
Critical weaknesses
  1. Layout and Presentation, Accuracy of presented results.

The main result table needs new baselines and benchmarks. Autoformer and DLinear are no longer strong performers in multivariate forecasting. Consider recent baselines like TimeMixer++, from 2025 onwards, or perhaps later too. Baselines excelling at spatio-temporal tasks would also be helpful if coupled with additional benchmarks reflecting these properties. Currently, the benchmarks are rather weak in terms of spatial/physical dependencies, which, after all, was a central concern in the opening paragraph. Consider adding traffic forecasting benchmarks, or any analogous benchmark with these dependencies. For example, line 275 shows that datasets contain 7,7,7,7,21 and 321 variables.

Further, Figure 2 is very hard to read; the labels are so small! Consider summarising this information within a single table. I would expect the datasets therein to also match the table above it. Overall, the presentation of the results can be greatly improved, e.g. inconsistent sizes of table font and column spacings, etc.

The table and figure of Table 2, Figure 3 (which contains two subplots) encodes redundant information. These items are representable within a single table, but currently take up a very large portion of the page. The paper does not have sufficient analysis to fill the full nine pages. I would expect the metrics explanation to be in the appendix (~ line 304).

On line 42. I don't believe the citation matches the statement. The statement is on using variables external to the dataset to improve forecasting accuracy. The citation is on emulating spatio-temporal dynamics of physical processes through deep learning, and was an early position paper in the emerging field. If these two concepts are truly connected, the prior statements are insufficient in their current state to explain why

  1. Understandable of the method section and theoretical results

Note on the proposition, approx line 216: It is hard to decipher if these four statements are related to each other. If they are, it needs to be clearer, otherwise they should be separated. Moreover, the results are not rigorously stated. The proposition should state such contents to mathematical precision without overcomplicating definitions. A similar statement applies to the assumptions, though they are sometimes rigorously stated.

A note on the writing: The prose of the method section is very difficult to understand. This couples with the very dense mathematical notation, which appears excessive and unnecessary. The prose is in an active voice, yet follows the tone of a mathematical derivation, which is not appropriate for a methodology section. The writing explains what is being done, but not the importance of each component to the CausalExoFormer. A good methodology should explain the synergy between components and how this solves a critical problem. At the moment, the writing makes the method appear incremental.

Questions for authors
  1. The paper does not properly synthesise the previous work, in the introduction or related work, in a way that makes it apparent what open problem is being addressed. Can the authors please clarify this point? Im am curious as to how mixing of exogenous variables with DAG constraints differs, and what the expected advantages are, before I can fully grasp the later parts of the paper.
Reviewer_QTdmOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes CausalExoFormer for forecasting with external time series. It learns which external variables to use, which past time points to use, and how information moves between variables. A shared graph serves all forecast targets, while each target's own history is kept out of its external input path. The model has the best average MAE on all six standard benchmarks and the best average MSE on four of them. Ablations and a synthetic experiment test the value of the source and lag selection.

Critical strengths
  1. The model gives its source and lag scores a clear role in prediction. Closing a source gate removes that source's direct and graph-mediated paths. The paper proves this property. The shared graph also avoids fitting a separate graph for every forecast target.
  2. The evaluation covers different settings. The paper reports full horizon results on six standard datasets and six lake tasks. The ablations show that learned source and lag selection helps. The synthetic test also checks whether the model finds the target-driving sources and their delays.
Critical weaknesses

The forecasting gains over the strongest baseline need an uncertainty check. CrossLinear is better on ETTh1 and ETTm1 MSE. On the lake tasks, some horizon-level gains are only 0.002 MAE, and the CrossLinear lake results come from single runs. Most standard benchmark baselines are taken from published tables. Without repeated runs under the same setup, it is hard to tell whether these small gains are reliable.

Questions for authors
  1. The synthetic SCM contains a known directed graph among the source variables, and CausalExoFormer learns a directed source graph B. Given the paper’s causal-structure claim, please clarify whether B is expected to recover the graph used to generate the data. If so, please explain how edges in B correspond to edges in the true graph, and report graph recovery results, such as edge precision and recall. Table 3 evaluates source selection and lag recovery for the target, but does not show whether the learned source graph matches the known graph.
  2. Could the authors rerun CausalExoFormer and the strongest baselines using the same data splits, preprocessing, and evaluation pipeline? Please report the mean and standard deviation over several random seeds. This would make the performance differences easier to assess.
  3. I found parts of the paper difficult to follow. Some sentences and words are not commonly used, and several symbols need clearer definitions. For example, what is k in equation 2? Please also briefly explain DYNOTEARS and PCMCI+ near Table 3, or point readers to the relevant discussion in Related Work. If AI tools were used to prepare the manuscript, please include the AI USE STATEMENT.
  4. Figure 2 shows a consistent V-shaped MSE curve: the lowest MSE occurs at H=192H=192 for all six lake tasks and all methods rather than error accumulation. In Appendix Table 8, MSE falls from H=96H=96 to H=192H=192, while MAE rises. Could the authors explain this pattern?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.