- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PTCD, a pretraining framework for time-series causal discovery. The method combines iterative attention for modeling intra and inter window temporal dependencies, context-conditioned Gaussian mixture modeling for exogenous variables, and causality-aware pretraining objectives involving synthetic interventions and latent-space causal mixup.
The model is pretrained on synthetic time series with known causal graphs and subsequently applied to real world causal discovery and root cause identification tasks, supporting both zero-shot inference and finetuning. Experiments on CausalRiver, SWaT, and MSDS demonstrate improvements over several classical and neural causal discovery baselines.
Critical strengths
- This is a meaningful application of pretrained causal discovery to temporal data. The framework integrates multi-scale temporal modeling with explicit exogenous noise estimation, addressing practical challenges in time-series causal discovery. The combination of intra window and inter-window attention is reasonable for capturing dependencies at different temporal scales.
- Promising empirical performance and computational efficiency. PTCD achieves competitive results on causal discovery and root cause identification benchmarks. The ablation studies suggest that the proposed components provide measurable benefits. In addition, the reported zero-shot inference efficiency highlights the potential advantage of amortizing causal discovery over a synthetic pretraining distribution.
Critical weaknesses
-
Limited novelty. The pretraining paradigm has already been explored in ACD (Löwe et al., 2022), AVICI (Lorch et al., 2022), and Causal Pretraining (Stein et al., 2024). I understand PTCD mainly extends this paradigm with additional architectural components and training objectives and it is acceptable, but the methodological novelty could be better explained, and comparisons with existing pretrained causal discovery methods can be strenghthed.
-
The paper lacks systematic evaluations on unseen mechanism families and scaling experiments. Although PTCD demonstrates promising zero-shot performance, its pretraining data are generated from a relatively restricted family of synthetic causal mechanisms. However, the robustness of PTCD to different causal mechanisms is not systematically investigated. In particular, controlled evaluations on held-out functional families or noise distributions would help clarify the extent of its generalization capability. Additionally, scaling experiments would strengthen the motivation for the proposed pretraining framework.
I will be happy to raise the score if those weaknesses can be addressed.
Löwe et al., 2022: Amortized Causal Discovery: Learning to Infer Causal Structure from Time-Series Data.
Lorch et al., 2022: Amortized Inference for Causal Structure Learning.
Stein et al., 2024: Embracing the Black Box: Heading Towards Foundation Models for Causal Discovery from Time Series Data.
Questions for authors
See above.