acceptodds
Withdrawn from ICLR 2027

CSALT: Training-Free Causal Skeleton Learning with Frozen Tabular Foundation Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learning causal skeletons from observational data is a fundamental yet challenging task in causal discovery. Constraint-based methods may lose power when data are scarce; continuous-optimization methods require per-dataset refitting; and pretrained causal discovery models require large-scale pretraining on simulations labeled with true graphs. Tabular foundation models offer another source of structural information for causal skeletons, but their attention gives only relative rankings and no statistical rule for determining edge existence. To bridge this gap, we introduce CSALT (Causal Skeleton Learning via Attention-Led Testing), which turns attention from a frozen tabular foundation model into the weights of a target-wise weighted Benjamini-Hochberg procedure. Permuting the target column on a disjoint split gives each candidate a permutation score, and an OR rule combines the target-wise rejection sets into an undirected skeleton. CSALT keeps all model parameters frozen and does not require additional pretraining with ground-truth graph supervision. We theoretically derive a sufficient condition involving attention ranking margins for excluding false edges with finitely many permutations, and further conditions for exact skeleton recovery. Empirical results on nonlinear synthetic data show that CSALT attains higher skeleton precision and F1 than all evaluated baselines. On two real datasets, our method achieves the highest F1 on UF and ties for the highest F1 on Sachs.

Primary area:
causal reasoning
Keywords:
causal skeleton learning, tabular foundation models, feature attention

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_bbcROpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes to use the attention weights of Tabular Foundation Models (TFMs) for causal discovery. The core contribution is to utilize a weighted Benjamini Hochberg (BH) procedure based on the attention scores from TFMs. They provide a theoretical analysis of their approach and test the performance of the proposed approach on a synthetic case-study as well as the Sachs and UF dataset.

Critical strengths

I find the idea to use the attention scores of frozen TFMs simple yet very interesting and generally promising since better TFMs will most likely also contribute to an improvement in the proposed method.

Critical weaknesses

My major concerns are in terms of soundness: the synthetic evaluation is on only 50 (!) SCMs. (It should have been very easy to extend the scope there). Further, the authors propose their own synthetic simulator and do not consider already proposed benchmarks or evaluation schemes (e.g. from other papers). They do evaluate on two real-world datasets, but this is not a lot either. I do have hope, however, that those concerns might be addressable in the rebuttal. A suitable causal discovery benchmark might be [1] and more real-world dataset (e.g. from CausalChambers [2]) could also be beneficial. Alternatively, the synthetic simulator in [3] that has been at least justified in a predictive setting could be a further alternative.

Another weakness is that the mathematical analysis is very hard to parse, both in the main text and the appendix.

[1] Göbler, Konstantin, et al. "$\texttt {causalAssembly} $: Generating Realistic Production Data for Benchmarking Causal Discovery." Causal Learning and Reasoning. PMLR, 2024.

[2] Gamella, Juan L., Jonas Peters, and Peter Bühlmann. "Causal chambers as a real-world physical testbed for AI methodology." Nature Machine Intelligence 7.1 (2025): 107-118.

[3] Reuter, Arik, et al. "Use What You Know: Causal Foundation Models with Partial Graphs." ICML 2026.

Questions for authors

Why do the authors evaluate on only 50 synthetic SCMs?

Reviewer_BFcEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

In the paper the authors propose a CSALT (Causal Skeleton Learning via Attention-Led Testing) method for learning causal skeletons from observational data using frozen tabular foundation models ((specifically LimiX and TabPFN v2.5)). The key idea is to combine the attention scores of a pretrained model with the permutation-based testing to identify candidate edges, without additional model training.

The proposed CSALT first splits the observational dataset into two disjoint subsets, using one subset to obtain attention-based scores, and the other to compute permutation scores by permuting the target variable. Then the method applies a weighted Benjamini–Hochberg procedure to select candidate neighbors for each target and combines the resulting selections using an OR rule to construct an undirected skeleton.

Critical strengths
  1. the paper is easy to follow and understand
  2. a simple approach to causal discovery by leveraging pretrained tabular models
  3. conducting experiments for validing the authors' claims
Critical weaknesses
  1. In the paper, the authors first divide the dataset into two separation subdatasets, then use attention-based scores and permutation scores on the two subdatasets to judge the edges between two variables. The idea is too simple and the rationale behind this idea for determining causal adjacency is not sufficiently convincing. In addition, to judge the edges between two variables, it seems that the strategy does not consider the remaining variables. Then it is hard to validate whether this strategy is correct. In addition, the attention scores reflect the model's predictive computation, not necessarily direct causal influence. In this paper, the authors need to provide stronger justification, either theoretically or empirically, for why these scores are informative for causal skeleton learning.

  2. The authors divide the dataset into two subsets, one to compute attention-based scores and the other to calculate permutation scores. However, the motivation for this design choice is not sufficiently explained. The paper should clarify why this separation is necessary and what theoretical or practical benefits it provides. It would also be useful to compare this design with alternatives that use the full dataset or different splitting strategies, and to quantify the potential impact of splitting on statistical power and edge recovery.

  3. It seems that the theorems in Section 5 do not provide a direct relations to the proposed method. It is not clear how these results establish that the attention scores produced by a frozen tabular foundation model are informative about causal adjacency, or why combining these scores with permutation-based testing leads to reliable skeleton recovery.

Questions for authors

See Critical Weaknesses

Reviewer_KgkiOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CSALT, a training-free method for causal skeleton discovery using frozen tabular foundation models. It extracts attention scores to construct candidate-edge weights, computes permutation scores on a disjoint data split, and applies a weighted Benjamini–Hochberg procedure for edge selection. An OR rule combines target-wise selections into an undirected skeleton. Based on CSALT, the authors further derive sufficient conditions for false-edge exclusion and exact skeleton recovery.

Critical strengths
  1. The paper explores the use of pretrained tabular foundation models for causal discovery, which offers an alternative to conventional causal discovery methods and pretrained graph prediction models.
  2. CSALT achieves the highest skeleton F1 score among the evaluated baselines on synthetic datasets. On real-world datasets, the LimiX-based model achieves the best F1 score on UF and ties for the highest F1 score on Sachs.
Critical weaknesses
  1. The performance of CSALT varies substantially across pretrained backbones. F1 score on UF decreases from 0.7166 with LimiX to 0.0430 with TabPFN v2.5, making it difficult to assess the robustness of the reported improvements.
  2. The permutation procedure tests whether the target variable is independent of all inputs, which does not directly correspond to causal nonadjacency. The weighted BH procedure does not generally guarantee false discovery rate control for skeleton edges, raising concerns about the statistical validity of the proposed edge selection procedure.
  3. The theoretical guarantees rely on attention-ranking margins defined using the true causal skeleton, which is unknown in practical applications. These conditions cannot directly verify the correctness of the recovered skeleton.
  4. The finite-permutation guarantee relies on a lower bound on permutation scores, which may also prevent true edges from being selected. The paper should further investigate this trade-off and clarify whether meaningful theoretical guarantees can be maintained under the improved testing procedure.
Questions for authors

Please refer to the weaknesses listed above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.