- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes to use the attention weights of Tabular Foundation Models (TFMs) for causal discovery. The core contribution is to utilize a weighted Benjamini Hochberg (BH) procedure based on the attention scores from TFMs. They provide a theoretical analysis of their approach and test the performance of the proposed approach on a synthetic case-study as well as the Sachs and UF dataset.
Critical strengths
I find the idea to use the attention scores of frozen TFMs simple yet very interesting and generally promising since better TFMs will most likely also contribute to an improvement in the proposed method.
Critical weaknesses
My major concerns are in terms of soundness: the synthetic evaluation is on only 50 (!) SCMs. (It should have been very easy to extend the scope there). Further, the authors propose their own synthetic simulator and do not consider already proposed benchmarks or evaluation schemes (e.g. from other papers). They do evaluate on two real-world datasets, but this is not a lot either. I do have hope, however, that those concerns might be addressable in the rebuttal. A suitable causal discovery benchmark might be [1] and more real-world dataset (e.g. from CausalChambers [2]) could also be beneficial. Alternatively, the synthetic simulator in [3] that has been at least justified in a predictive setting could be a further alternative.
Another weakness is that the mathematical analysis is very hard to parse, both in the main text and the appendix.
[1] Göbler, Konstantin, et al. "$\texttt {causalAssembly} $: Generating Realistic Production Data for Benchmarking Causal Discovery." Causal Learning and Reasoning. PMLR, 2024.
[2] Gamella, Juan L., Jonas Peters, and Peter Bühlmann. "Causal chambers as a real-world physical testbed for AI methodology." Nature Machine Intelligence 7.1 (2025): 107-118.
[3] Reuter, Arik, et al. "Use What You Know: Causal Foundation Models with Partial Graphs." ICML 2026.
Questions for authors
Why do the authors evaluate on only 50 synthetic SCMs?