1. The foundations and empirical validation of the claimed causal structure learning capability are insufficient.
The paper aims to recover causal structures, including latent confounding, from observational data using an amortized model. This is already a highly challenging research problem on its own. In particular, learning ADMGs involves different graphical representations, identifiability assumptions, and equivalence classes. Even with unlimited observational data, the underlying causal structure is generally not uniquely identifiable without additional assumptions.
The paper supervises predictions of adjacency, ancestry, and confounding matrices using labels from simulated SCMs. However, it is unclear what aspects of these structures are identifiable under the assumed data-generating processes, how the learned predictions respect graphical consistency constraints, or whether their uncertainties meaningfully reflect observational non-identifiability. Predicting each structural relation with an elementwise loss is not necessarily equivalent to learning a coherent posterior over causal graphs.
These questions matter because the paper claims substantially more than predictive accuracy on simulated graph labels. It presents the model as a general-purpose approach to causal structure learning, including latent confounding. Establishing such a capability would require a much more systematic characterization of the assumptions under which it succeeds, the structures it can reliably recover, and its behavior when those structures are not identifiable.
The empirical evidence is also insufficient for this claim. Most evaluations use synthetic SCMs drawn from the authors' training prior. Although the paper includes an OOD experiment with random Fourier mechanisms, this remains a limited test of generalization across the much broader space of causal graphs, functional mechanisms, noise distributions, and latent confounding structures. The real-data evaluation is restricted to two datasets and does not provide ground-truth confounding information.
Related work that would help clarify the problem formulation and evaluation includes Learning Identifiable Structures Helps Avoid Bias in DNN-based Supervised Causal Learning (Zhang et al., 2025), Causal Discovery with Unobserved Confounding and Non-Gaussian Data (Wang and Drton, 2023), and Causal Identification under Markov Equivalence: Completeness Results (Jaber et al., 2019).
2. The outcome prediction claims are not sufficiently supported, particularly for counterfactual inference and the proposed unification of the three tasks.
The paper brings causal structure learning, interventional prediction, and counterfactual prediction into one pretrained network. I find the overall direction interesting, but each of these tasks has substantial unresolved challenges, and the current evaluation does not convincingly establish that the proposed model can address them in a generalizable manner.
One fundamental concern is the counterfactual prediction setting. Section 5.3 and Appendix D.2 appear to supply the model with counterfactual covariate values generated by the underlying SCM, including variables recomputed after the treatment intervention. Such values would generally not be available when answering a real counterfactual query. Conditioning on them is different from inferring a counterfactual outcome using only factual observations. This distinction needs to be resolved before the reported counterfactual performance can be interpreted as evidence of the claimed capability.
The real-data experiments do not fully address this issue. Although the observational datasets are real, the interventional and counterfactual evaluation targets are generated using DoWhy models. Agreement with these targets does not independently validate the model's predictions against real causal outcomes.
I also find the methodological contribution of jointly learning structure and outcomes insufficiently established. The model shares representations across the tasks, but its predicted ADMG is not explicitly used as an intermediate input to outcome prediction when no graph is supplied. The structure-loss ablation shows modest improvements in interventional and counterfactual prediction, but this is also consistent with the benefits of ordinary auxiliary-task regularization. It does not establish that the model has learned a structure-based causal inference procedure, or clarify when structure learning is necessary for estimating the target causal effect.
More broadly, the evaluation of supervised causal learning needs to distinguish genuine causal generalization from statistical regularities introduced by synthetic data generators. The concerns raised in Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy to Game (Reisach et al., 2021) are particularly relevant here.
Overall, the paper demonstrates that several causal prediction objectives can be learned jointly under a synthetic training distribution. However, I do not find sufficient evidence that this constitutes a reliable or general-purpose foundation model for causal discovery, intervention, and counterfactual inference.
We appreciate your feedback and the time you took to review our paper. We are glad that you found our work clear and informative, and we appreciate your positive comments regarding the strengths of our model.
Compute times comparison
Compared to the outcome prediction methods, our method uses a similar amount of time to make predictions. On our H100 GPU, all methods make predictions in under 100ms / example (10 samples per second) in our tests with up to 1024 observations. This is because all models are tabular transformers with similar sizes and setup. The overhead of our graph embedding layer is only a few ms per example.
Compared to the structure prediction, our method is again similar in speed compared to AVICI and Arrow which are tabular transformers. The classical baselines use the causal-learn package and are run on the cpu. With 1024 observations on a 16 core CPU. LiNGAMICA (52ms / example) and PC (760ms / example) are faster than GES (3.7s / example) and FCI (10s / example). Apart from LiNGAMICA, the other methods are substantially slower than the neural methods.
N-ADMG
We evaluated N-ADMG on the same out-of-distribution Fourier-feature dataset with 1024 observations as used in the main paper. The TabPFN-CFM outcome results below are those reported in Table 1.
Graph prediction (edge accuracy in %, AUROC; higher is better)
Outcome MSE (lower is better)
TabPFN-CFM has substantially better accuracy for outcome prediction and graph prediction than N-ADMG. We attribute this to our method's PFN setting, where we are less likely to overfit on the relatively small datasets as well as our refined training procedure. Furthermore, N-ADMG requires training a new model for each dataset, which is much slower, at around 5 minutes per dataset compared to TabPFN-CFM which takes around 0.1 second.