acceptodds
Withdrawn from ICLR 2027

Time Series Causal Discovery via Context-Conditioned and Causality-Augmented Pretraining

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Causal discovery from time series is critical for many real-world applications, such as tracing the root causes of anomalies. Existing approaches typically rely on dataset-specific optimization, making it difficult to transfer their causal discovery capabilities to new time series governed by diverse causal mechanisms. In this paper, we propose PTCD, a novel Pretraining framework for Time-series Causal Discovery, which improves cross-task generalization through context-conditioned modeling and transferable causal augmentation. To model complex temporal causal dependencies, PTCD employs a dual-scale iterative attention mechanism to capture window-level causal relationships, and a Gaussian mixture with a context-level routing mechanism to handle heterogeneous exogenous distributions. To further address distribution shifts across causal graphs, PTCD adopts a pretraining paradigm on synthetic datasets that integrates intervention-based learning and a causal mixup strategy, promoting stable causal discovery and stronger generalization. Extensive experiments on multiple real-world out-of-distribution (OOD) datasets demonstrate that PTCD excels in both causal discovery and root cause identification.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Causal Discovery, Root Cause Identification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DjJUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PTCD, a pretraining framework for time-series causal discovery. The method combines iterative attention for modeling intra and inter window temporal dependencies, context-conditioned Gaussian mixture modeling for exogenous variables, and causality-aware pretraining objectives involving synthetic interventions and latent-space causal mixup.

The model is pretrained on synthetic time series with known causal graphs and subsequently applied to real world causal discovery and root cause identification tasks, supporting both zero-shot inference and finetuning. Experiments on CausalRiver, SWaT, and MSDS demonstrate improvements over several classical and neural causal discovery baselines.

Critical strengths
  1. This is a meaningful application of pretrained causal discovery to temporal data. The framework integrates multi-scale temporal modeling with explicit exogenous noise estimation, addressing practical challenges in time-series causal discovery. The combination of intra window and inter-window attention is reasonable for capturing dependencies at different temporal scales.
  2. Promising empirical performance and computational efficiency. PTCD achieves competitive results on causal discovery and root cause identification benchmarks. The ablation studies suggest that the proposed components provide measurable benefits. In addition, the reported zero-shot inference efficiency highlights the potential advantage of amortizing causal discovery over a synthetic pretraining distribution.
Critical weaknesses
  1. Limited novelty. The pretraining paradigm has already been explored in ACD (Löwe et al., 2022), AVICI (Lorch et al., 2022), and Causal Pretraining (Stein et al., 2024). I understand PTCD mainly extends this paradigm with additional architectural components and training objectives and it is acceptable, but the methodological novelty could be better explained, and comparisons with existing pretrained causal discovery methods can be strenghthed.

  2. The paper lacks systematic evaluations on unseen mechanism families and scaling experiments. Although PTCD demonstrates promising zero-shot performance, its pretraining data are generated from a relatively restricted family of synthetic causal mechanisms. However, the robustness of PTCD to different causal mechanisms is not systematically investigated. In particular, controlled evaluations on held-out functional families or noise distributions would help clarify the extent of its generalization capability. Additionally, scaling experiments would strengthen the motivation for the proposed pretraining framework.

I will be happy to raise the score if those weaknesses can be addressed.

Löwe et al., 2022: Amortized Causal Discovery: Learning to Infer Causal Structure from Time-Series Data.

Lorch et al., 2022: Amortized Inference for Causal Structure Learning.

Stein et al., 2024: Embracing the Black Box: Heading Towards Foundation Models for Causal Discovery from Time Series Data.

Questions for authors

See above.

Reviewer_5U5qOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PTCD, a pretraining framework for time-series causal discovery. A model with alternating intra- and inter-window attention predicts a lagged causal graph from node embeddings, while a context-conditioned mixture-of-Gaussians module estimates exogenous noise used for reconstruction and for root-cause scoring; an intervention pretext task and a latent causal mixup are added during pretraining on synthetic data. The model is fine-tuned without graph labels and evaluated on causal graph discovery over river-flow networks and on root-cause identification in two industrial testbeds, and is claimed to outperform classical, deep, and pretrained baselines.

Critical strengths

S1. One pretrained model serves two tasks: The estimated exogenous variables feed both graph reconstruction and root-cause scoring, and the ablation shows this component carries the root-cause result (App. Table 6).

S2. Reproducibility path: Anonymous code and data links, an efficiency table, and hyper-parameter sensitivity curves are provided.

Critical weaknesses

W1. The key baseline and the classical rows are of unclear provenance. The paper does not say whether Causal Pretraining [1] was run zero-shot or fine-tuned on Bavaria with graph labels, and describes it as supervised in Sec. 2 but as self-supervised in App. B.2. The CausalRivers paper [2] reports zero-shot CP at about 0.63–0.64 mean AUROC, whereas the CP-Transformer row here (0.80) matches that paper's label-supervised fine-tuned CP in 8 of 10 subsets; the CP-GRU row has no counterpart. The VAR, VarLiNGAM and PCMCI rows of Table 2 are identical cell for cell to CausalRivers Table 3, with discrepancies in CDMI and DYNOTEARS, and no source is given. No baseline hyper-parameters or protocols are described.

W2. Evaluation does not follow the field. Causal discovery is tested on a single real dataset family and a self-built synthetic set whose mechanisms are MLP black boxes with no stated dynamical equations; none of the standard benchmarks (Lorenz-96 [3], fMRI NetSim [4], DREAM3, finance, CausalTime [5]) is used. No method after 2024 is compared, and Neural Granger [6], JRNGC [7], and UnCLe [8] are absent. Root-cause identification is compared only with causality-based methods, not with non-causal root-cause analysis or anomaly detection.

W3. Claims exceed the evidence. The transferability claim rests on results after fine-tuning (zero-shot 0.74, App. Table 7), although the fine-tuning objective (Eq. 14) contains no graph supervision and the paper does not explain how it changes the graph head. Theorem 4.1 is the GAN density-ratio identity under a fixed single-target intervention and does not establish identifiability or correspond to the window-level loss of Eq. 10; under the paper's own assumptions the lagged graph is already identifiable, so the Markov-equivalence motivation does not apply. Theorem 4.2 is a heuristic sketch whose appendix treats a dense Jacobian and a squared loss that differ from the binary adjacency and BCE actually trained.

W4. Statistical reporting. Tables 2 and 4 report no variance; ablation margins are 0.02–0.04. Table 3 uses three decimals with standard deviations while Tables 2 and 4 use two decimals without. App. Table 6 contains an impossible value (Avg@10 = 0.492 for "w/o inter", inconsistent with its own Recall@K). RCD and CIRCA have identical rows on SWaT, and the seed count is never stated.

W5. No causal findings. No discovered graph is shown or analysed; all evidence is aggregate AUROC and Recall@K.

W6. Presentation. Table 1 is unreadable at print size. Notation is inconsistent: G̃_p in Eq. 8 is undefined, Eq. 14 contradicts the stated reconstruction loss, Eq. 9 writes the KL in the reverse direction, and "AC@1" is referenced but never defined.

References

[1] Embracing the Black Box: Heading Towards Foundation Models for Causal Discovery from Time Series Data. Stein et al., AAAI Workshop on AI for Time Series, 2024.

[2] CausalRivers: Scaling up Benchmarking of Causal Discovery for Real-World Time-Series. Stein et al., ICLR, 2025.

[3] Extensive chaos in the Lorenz-96 model. Karimi and Paul, Chaos, 2010.

[4] Network modelling methods for FMRI. Smith et al., NeuroImage, 2011.

[5] CausalTime: Realistically Generated Time-series for Benchmarking of Causal Discovery. Cheng et al., ICLR, 2024.

[6] Neural Granger Causality. Tank et al., IEEE TPAMI, 2021.

[7] Jacobian Regularizer-based Neural Granger Causality. Zhou et al., ICML, 2024.

[8] UnCLe: Towards Scalable Dynamic Causal Discovery in Non-linear Temporal Systems. Bi et al., NeurIPS, 2025.

Questions for authors
  1. (W1) Were CP-Transformer and CP-GRU run zero-shot or fine-tuned on Bavaria with graph labels? Were the VAR, VarLiNGAM, PCMCI, CDMI and DYNOTEARS rows of Table 2 taken from the CausalRivers paper, and if so, why do some cells differ?
  2. (W3) How does fine-tuning with reconstruction and KL only change the output of the graph head? Please report zero-shot and fine-tuned PTCD per subset.
  3. (W3) Under the stated assumptions the lagged graph is identifiable from observational data; what is the theoretical role of the intervention task, and how does Theorem 4.1 relate to the window-level loss of Eq. 10?
  4. (W4) How many seeds underlie Tables 2–4, and what explains the Avg@10 value in App. Table 6 and the identical RCD/CIRCA rows?
Reviewer_yYBtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PTCD, a pretraining framework for time-series causal discovery and root-cause identification. It combines alternating intra-window and inter-window attention with context-conditioned Gaussian-mixture modeling of exogenous variables. Training on synthetic time series uses causal-graph supervision, reconstruction, intervention detection with contrastive alignment, and mixup of latent representations and graph labels. The pretrained model is then adapted using reconstruction and KL objectives. Evaluation includes causal discovery after fine-tuning on Bavaria and testing on Eastern Germany, and root-cause identification on MSDS and SWaT. The reported results favor the complete system over the evaluated baselines, with additional component ablations and a zero-shot efficiency comparison.

Critical strengths
  1. A coherent framework for a useful transfer problem. The paper targets the cost of optimizing a causal-discovery model separately for each dataset. Its representation learning, contextual exogenous-variable modeling, and training augmentations have identifiable roles in the proposed transfer pipeline. Linking the estimated exogenous disturbances to root-cause ranking also makes the framework relevant beyond graph prediction. The contribution has value as an integrated learning system.

  2. Encouraging empirical evidence across geography and downstream tasks. PTCD is fine-tuned on Bavaria and tested on Eastern Germany, where it obtains mean AUROC 0.85 versus 0.80 for CP-Transformer (Table 2). It also improves the reported root-cause ranking metrics on MSDS and SWaT (Table 3). Tables 4 and 6 support the usefulness of the individual components, particularly exogenous-variable modeling and intervention-based training. These results provide a substantive empirical basis for further investigation.

Critical weaknesses
  1. The central benefit of pretraining is not adequately isolated. The manuscript introduces a new architecture, synthetic graph supervision, augmentation, and downstream adaptation together, but does not report the same PTCD architecture trained from random initialization without synthetic pretraining. The component ablations in Tables 4 and 6 do not replace this control. Likewise, Table 7 compares zero-shot and fine-tuned versions of an already pretrained model, so it measures the benefit of adaptation rather than pretraining. The corpus, supervision, and adaptation protocol used for CP are not specified clearly enough to assess whether the comparison isolates the proposed advances. The results support the complete system, but the source of its improvement remains uncertain. This is consequential because transferable causal pretraining is the paper's central contribution.

  2. The causal-identification and stability guarantees exceed the derivations. In Theorem 4.1 and Appendix A.1, the optimal intervention discriminator recovers a conditional density ratio under the stated intervention construction. This does not by itself establish that the learned graph head must recover the unique true parent set. For Theorem 4.2, Appendix A.2 claims that low mixup loss penalizes the predictor's Jacobian norm and bounds its local Lipschitz constant. A scalar probability predictor G(s) = s / delta on [0, delta], with matching endpoint targets, has zero interpolation loss in Eq. (21), but derivative 1 / delta, which can be arbitrarily large. Assumption A.2 does not supply a quantitative scale or slope constraint that excludes this example. Interpolation consistency therefore does not establish the claimed gradient-norm control. These gaps weaken the theoretical justification, although they do not invalidate the empirical gains from the augmentations.

Questions for authors
  1. Pretraining control. Could you report a randomly initialized PTCD model with the same architecture, downstream data, adaptation objective, and optimization budget as the pretrained model? Results on both causal discovery and root-cause identification would directly clarify the contribution of synthetic pretraining.

  2. Baseline protocol. For the CP variants in Table 2, what synthetic corpus, supervision, model capacity, training budget, hyperparameter-selection procedure, and Bavaria adaptation protocol were used? Which conditions were matched to PTCD? Please also reconcile the description of supervised graph training in Section 2 with masked-subseries reconstruction in Appendix B.2.

  3. Scope of the guarantees. What additional argument connects the optimal discriminator in Theorem 4.1 to recovery by the actual graph head? Can Theorem 4.2 be corrected to address the affine-predictor example and the BCE objective actually optimized, rather than only the squared-error surrogate in Eq. (21)? Alternatively, would you narrow the statements to the guarantees that the derivations establish?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.