acceptodds
Withdrawn from ICLR 2027

Pretrained Causal Discovery at Any Size

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pretrained causal discovery from observational data trains a model on simulated systems and applies it to new data in a single inference pass. In the temporal case, most such models read the raw time series, tying them to the scale of the training data and restricting flexibility. A different strategy, which began with supervised cause-effect classification over features of a sample, turns discovery into supervised learning over features of the data rather than over the data itself (D2C), and and was recently extended to time series (TD2C). We present a deep learning reformulation of that strategy, proposing an architecture that works with a single tensor holding a family of statistics, at the level of every target, source and lag triplet. For this reason, its representation is not tied to a particular number of variables or lags, and when evaluated against its pairwise predecessor and against classical discovery algorithms, it matches or exceeds both in accuracy while scaling far better with the number of variables.

Primary area:
causal reasoning
Keywords:
causal discovery, time series, amortized inference, pretrained models, supervised causal learning, feature-based causal inference

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_JPkwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a permutation-equivariant neural architecture for pretrained temporal causal discovery. Instead of operating directly on raw time series, the method constructs a tensor of dependence statistics indexed by target, source, and lag, and jointly predicts the causal graph. Since the model parameters do not depend on the number of variables or lags, a single pretrained model can be applied to systems of different sizes. The method is evaluated on several simulated and benchmark datasets against classical causal discovery methods, descriptor-based methods, and existing pretrained approaches.

Critical strengths
  1. The architecture removes a practical size restriction of existing pretrained models. Parameterizing the model over dependence-statistic tensors rather than raw time series allows the same network to process different numbers of variables and lags. 59211(4)
  2. The method performs strongly on several simulated benchmarks. In particular, the reported performance on the internal paired experiments and CDML indicates that joint processing of dependence statistics can be effective under the evaluated simulation distributions.
Critical weaknesses
  1. Limited novelty. The method is largely a permutation-equivariant neural reformulation of existing descriptor-based causal discovery, and the methodological advance beyond size-agnostic joint processing is limited.

  2. “Any size” is stronger than what is demonstrated. The method is architecturally compatible with different system sizes, but this does not imply reliable causal-discovery generalization across sizes or distributions.

  3. Limited empirical evidence. The strongest results are concentrated in simulated settings, while performance on more realistic benchmarks is substantially less convincing.

  4. Unclear benefit of the neural formulation. The proposed model is not consistently better than simpler descriptor-based or classical methods, making the benefit of joint neural processing unclear.

  5. Weak presentation. The experimental organization, writing, and visualizations make the main empirical conclusions unnecessarily difficult to identify.

Questions for authors
  1. What is the main methodological novelty beyond a size-agnostic neural reformulation of descriptor-based causal discovery?

  2. What evidence shows that the method generalizes statistically, rather than merely accepting different input sizes?

  3. When and why does joint neural processing outperform the simpler per-edge formulation?

  4. Can the authors improve the evaluation on realistic data and simplify the presentation of the main results?

Reviewer_HzjtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a permutation-equivariant neural network for pretrained temporal causal discovery using dependence statistics. The method combines equivariant pooling, path composition, and lag mixing to handle varying numbers of variables and lags. Experiments are conducted on several synthetic and realistic benchmarks.

Critical strengths
  1. The problem is relevant, and the proposed framework is reasonably straightforward. 2. The paper includes comparisons with several existing causal discovery methods.
Critical weaknesses
  1. The method mainly combines existing dependence statistics with standard permutation-equivariant neural networks. The architectural contributions appear incremental, and the paper does not establish a sufficiently strong methodological innovation.

  2. The proposed method performs poorly on several benchmarks, particularly CausalTime and Kuramoto, where simple statistical measures or traditional methods substantially outperform it. These results raise concerns about its practical effectiveness and generalization ability.

  3. The paper lacks systematic ablations to demonstrate the individual contributions of equivariant pooling, path composition, and lag mixing. It is therefore unclear whether the proposed architectural components are necessary.

  4. Although the architecture supports variable input dimensions, the computational complexity remains high. The experiments do not sufficiently demonstrate scalability to substantially larger systems.

Questions for authors
  1. Can the authors provide ablation studies to demonstrate the necessity of equivariant pooling, path composition, and lag mixing?

  2. Why does the proposed method perform substantially worse than simple statistical baselines on CausalTime and Kuramoto? What explains its limited generalization?

  3. Can the authors provide stronger evidence supporting the "any size" claim, particularly for systems substantially larger than those encountered during training?

Reviewer_1LAROpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a size-agnostic pretrained model for time-series causal discovery. Each candidate lagged edge is encoded by a fixed set of dependence statistics, and a permutation-equivariant network with pooling, path-composition, and lag-mixing operators maps the resulting tensor to edge probabilities, so one model trained on synthetic systems can be applied to systems with any number of variables and lags. The method is evaluated against classical algorithms, its descriptor-based predecessor, and released pretrained models on synthetic, simulated, and one real dataset, and is claimed to be competitive while transfer to realistic data remains open.

Critical strengths

S1. Size-agnostic architecture: The weights are independent of the number of variables and lags, which removes a concrete limitation of released pretrained models (Causal Pretraining [9] is fixed at five variables, LCM [10] at twelve).

S2. Careful and honest protocol: All methods are scored on identical cells, seed ranges and prevalence are reported, and the (so-called) realistic-data failures are stated rather than hidden.

Critical weaknesses

W1. CausalTime is not a real-data endpoint. The paper presents CausalTime [1] as the realistic end of its evaluation, but CausalTime is generated data: a neural model is fitted to real series and the ground-truth graph is constructed by the benchmark pipeline, which its authors explicitly state is not the true causal graph. Rivers is therefore the only real dataset, and the model's AUROC there is below VAR-LiNGAM and TD2C.

W2. Standard benchmarks are missing. Lorenz-96 [2] and fMRI NetSim [3], used by essentially every neural time-series causal discovery paper, are absent, so results cannot be placed against the literature.

W3. Kuramoto is uninformative in the paper's favour and informative against it. Lasso-Granger reaches 0.97 and partial correlation 0.91, so the set is saturated by linear methods; the proposed model reaches only 0.74, below both.

W4. Synth_230K comparison is incomplete. No classical algorithm is run on this set, yet it is one of the three datasets on which the model is reported to lead.

W5. Deep baselines are omitted although published numbers exist. Neural Granger [4], TCDF [5], GVAR [6], CUTS/CUTS+ [11,13], JRNGC [8], and UnCLe [7] are not compared. The CausalTime benchmark itself reports 13 methods on the same data, with CUTS+ [11] and LCCM [12] around 0.8–0.9 AUROC versus 0.46–0.54 here, which places the model at the level of plain Granger causality or PCMCI. LCM [10], the closest competitor, could not be run.

W6. Results do not support the claim. The model leads only on synthetic sets (internal; CDML by 0.01–0.02; Synth_230K). On Kuramoto, CausalTime, Rivers, and TCDF (AUPRC) either a classical method or one of the model's own raw input statistics is better; on CausalTime it is below its own HSIC input.

W7. No ablation and no qualitative evidence. There is no ablation of the three operators or the descriptor channels, not even a per-cell MLP without coupling, so the in-distribution gains cannot be attributed to the architecture. No recovered graph, error analysis, or case study is shown; the paper consists of metrics only.

References

[1] CausalTime: Realistically Generated Time-series for Benchmarking of Causal Discovery. Cheng et al., ICLR, 2024.

[2] Extensive chaos in the Lorenz-96 model. Karimi and Paul, Chaos, 2010.

[3] Network modelling methods for FMRI. Smith et al., NeuroImage, 2011.

[4] Neural Granger Causality. Tank et al., IEEE TPAMI, 2021.

[5] Causal Discovery with Attention-Based Convolutional Neural Networks. Nauta et al., Machine Learning and Knowledge Extraction, 2019.

[6] Interpretable Models for Granger Causality Using Self-explaining Neural Networks. Marcinkevičs and Vogt, ICLR, 2021.

[7] UnCLe: Towards Scalable Dynamic Causal Discovery in Non-linear Temporal Systems. Bi et al., NeurIPS, 2025.

[8] Jacobian Regularizer-based Neural Granger Causality. Zhou et al., ICML, 2024.

[9] Embracing the Black Box: Heading Towards Foundation Models for Causal Discovery from Time Series Data. Stein et al., AAAI Workshop on AI for Time Series, 2024.

[10] Large Causal Models for Temporal Causal Discovery. Kougioulis et al., arXiv:2602.18662, 2026.

[11] CUTS+: High-dimensional Causal Discovery from Irregular Time-series. Cheng et al., AAAI, 2024.

[12] Latent Convergent Cross Mapping. De Brouwer et al., ICLR, 2021.

[13] CUTS: Neural Causal Discovery from Irregular Time-Series Data. Cheng et al., ICLR, 2023.

Questions for authors

Please see the weaknesses and clarify. In particular:

  1. (W7) Please provide ablations on internal, CDML, Kuramoto, and CausalTime: per-cell MLP without coupling; without path composition; without pooling; without lag mixing; and without the HSIC and correlation-ratio channels.
  2. (W6) Why does the model fall below its own HSIC input on CausalTime? Does a simple mean or max over the eight input statistics already exceed it there?
  3. (W4, W5) Please run the classical algorithms on Synth_230K, and compare against the published CausalTime results for CUTS+, LCCM, and Neural Granger.
  4. (W1) Which series and which block of the CausalTime ground truth were used, given that the benchmark's generated data contain instantaneous residual links that violate the paper's no-contemporaneous-effects assumption?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.