What Do Causal Discovery Foundation Models Learn?

Recovering a unique causal structure from observational data is difficult due to finite data and non-identifiability. Recent causal discovery foundation models pretrain on synthetic causal data and directly infer structures for new datasets, avoiding repeated structure search. Yet when the causal structure is not identifiable or the true data-generating distribution lies outside the distribution family used for pretraining, what these models learn and which structures they can recover remain unclear. To answer these questions, we develop a statistical framework that characterizes both the graph distribution learned through pretraining and its large-sample recovery target. We prove that, as the number of pretraining tasks and model capacity increase, the learned graph distribution approaches the posterior over graphs induced by the pretraining prior. As the sample size increases, this posterior concentrates on the KL-best-fitting graph set under model misspecification. Under correct specification and the Markov and faithfulness assumptions, it concentrates on the Markov equivalence class of the true DAG; under additional identifiability assumptions, it concentrates on the true DAG. These results provide a unified perspective for understanding existing models and guide the design of pretraining distributions and model architectures. As a concrete instantiation, we factorize DAGs into skeletons and topological orders, use conditional diffusion to model multimodal distributions over topological orders, and orient skeletons by sampled orders to guarantee acyclicity. Across diverse synthetic benchmarks, our method achieves stronger overall structure recovery than traditional methods and other causal discovery foundation models.