- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a new foundation model for causal inference. These foundation models answer causal queries about unknown causal systems given observational and interventional data from that system. The key novelty is that they include interventional data as context at test time.
Critical strengths
- Overall Excellent presentation
- Use of the real-world Chambers dataset.
- Natural and relevant extension of causal foundation models.
- Figure 3 and the discussion of identifiability are particularly useful.
Critical weaknesses
- The biggest weakness is the external CFM comparisons. They are difficult to interpret because, it seems, the baselines are pretrained under different priors and information regimes. CIDER-FM-Obs appears to be the only closely matched baseline.
- The novelty relative to existing foundation models using interventional context could be explained more clearly.
- It remains unclear how much of the gain comes from the proposed three-way attention and if this is really the best way of using the interventional context.
- The repository has no README explaining how to reproduce results of the manuscript.
Questions for authors
- Which parts of the theoretical results are specific to CIDER-FM?
- How does CIDER-FM relate to MapPFN (not cited), TabCausal (not cited), and PerturbPFN (cited but not discussed in detail)?
- Given some evidence that tabular foundation models can behave like implicit retrieval methods, how do you assess whether CIDER-FM learns transferable causal reasoning or mainly exploits similarity to its training prior and context?
- Could you include a comparison to a foundation model that can be considered fair?
- How do I reproduce the paper's results using the repo?
4 comments
Response to Reviewer duDL (2/3)
It remains unclear how much of the gain comes from the proposed three-way attention and if this is really the best way of using the interventional context.
We now report a controlled two-axis versus three-axis ablation in Appendix M (pp. 38-41), with closely matched parameter counts and the same pretraining and larger-context adaptation schedule. At 20 variables and 512 samples per regime, three-axis attention provides 1.41× and 3.75× higher training throughput with 5 and 19 regimes, respectively, although it offers no speedup with only 2 regimes. After adaptation, it also achieves a modest NLL improvement of 0.0130 (95% paired CI: [0.0037, 0.0229]), while differences in MSE and remain inconclusive. These results support the architecture’s computational advantage in larger multi-regime contexts, alongside a modest predictive benefit, but do not establish that it is universally optimal.
The repository has no README explaining how to reproduce results of the manuscript.
We apologize for forgetting to add the README and added a README with detailed instructions for reproducing our experiments.
Questions
Which parts of the theoretical results are specific to CIDER-FM?
To the very best of our knowledge, all theoretical results fo Causal Foundation models are with respect to a posterior (predictive), in our case, induced by the prior for the respective data-generating process, which the Causal Foundation Model learns to approximate. This includes the results in [1], [2], [3], and [4].
Our results are not different in that they are about the ''ground-truth'' causal posterior predictive , and are thus independent of and do not account for the inference method used to approximate it.
We also moved the details on our theoretical arguments to the Appendix B to D (pp. 15-18) since we see them not as novel theoretical constributions, but rather as simple formalisations of the general principles behind what makes CIDER-FM work and included them mainly for the purpose of building intuition.
How does CIDER-FM relate to MapPFN (not cited), TabCausal (not cited), and PerturbPFN (cited but not discussed in detail)?
MapPFN focuses on generating multivariate post-perturbation distributions for biological systems under the assumption of no unobserved confounders. CIDER-FM instead directly predicts conditional interventional densities , allowing hidden confounding and query-specific conditioning values. PerturbPFN addresses a complementary setting in which perturbation targets and strengths are unknown. It uses perturbation descriptors and observed responses to estimate a graph, targets, and strengths, then predicts responses through an SCM decoder with additive mechanism shifts. CIDER-FM assumes known intervention targets and assigned values, and directly approximates the predictive density without explicitly estimating a graph as an intermediate step.
TabCausal is a model that performs amortised causal discovery. Analogously to CIDER-FM, it is trained on mixed observational-interventional data, but outputs edge probabilities for causal graph instead of predicting a causal effect. TabCausal is also not trained with hidden confounding.
- Could you include a comparison to a foundation model that can be considered fair?
As detailed in our response to W1, we use three complementary foundation-model comparisons. CIDER-FM-Obs provides the most closely controlled comparison, sharing CIDER-FM’s architecture, model size, and training budget, while using observational context only. One can see CIDER-FM-Obs as version of DoPFN [1] with continuous treatment that matches exactly our training setup and architecture.
We also train a custion Graph4CFM model [4] to provides an external CFM comparison under CIDER-FM’s ComplexMech natural SCM prior, evaluated on the same queries with matched total context sizes. Its model size and training budget differ from ours. TabPFN provides a supplementary comparison to a general-purpose predictive foundation model.
How do I reproduce the paper's results using the repo?
Please refer to our updated code-base together with the new README.
Response to Reviewer duDL (3/3)
Given some evidence that tabular foundation models can behave like implicit retrieval methods, how do you assess whether CIDER-FM learns transferable causal reasoning or mainly exploits similarity to its training prior and context?
We see, for example, the gains of CIDER-FM compared to the purely predictive TabPFN (which was trained with much more compute) on our benchmarks that pose carefully controlled causal challenges as strong evidence that CIDER-FM performs causal reasoning. We see the faxct that the model generalizes to the real-world Causal Chambers data as evidence that those capabilities are transferrable, even to real-world scenarios.
We believe that obtaining explicit mechanistic insights into CIDER-FM's behaviour, similar to emerging work in the context of predictive Tabular Foundation Models [5], is challenging yet interesting direction for future work.
References
[1] Robertson, Jake, et al. "Do-pfn: In-context learning for causal effect estimation." Neurips 2025.
[2] Balazadeh Meresht, Vahid, et al. "Causalpfn: Amortized causal effect estimation via in-context learning." Neurips 2025.
[3] Ma, Yuchen, et al. "Foundation models for causal inference via prior-data fitted networks." ICLR 2026.
[4] Reuter, Arik, et al. "Use what you know: Causal foundation models with partial graphs." ICML 2026.
[5] Balef, Amir, et al. "Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Models". ICML 2026.
The Rebuttal Period Will Be Over Soon (Less Than 48 Hours Left)
Thank you again for your review!
We address your comments and questions in our answers above.
If you have further questions, we are very happy to answer them.
If everything is addressed, we would greatly appreciate you increasing your score.
Response to Reviewer duDL (1/3)
Thank you for your comprehensive review!
We are especially happy that you find the presentation of the paper to be excellent.
We adress your concerns and questions below and have updated the manuscript accordingly.
Weaknesses
Thank you for highlighting this distinction. In general, we are not aware of any directly applicable CFM that we can directly compare to for this task.
In repsonse to your comment, we trained a new Graph4CFM model [4] using CIDER-FM’s ComplexMech prior. The retraining is needed to make sure both CIDER-FM and the Graphs4CFM are trained on the same prior. Please note that we always provide the same number of context samples for both models. While CIDER-FM uses some interventional samples, the Graphs4CFM model uses observational data together with ground-truth graphical information.
ComplexMech: original 512 test SCMs
While, as expected, Graphs4CFM improves when we provide more graph information (full ancestry), CIDER-FM obtains lower MSE and higher mean, median, and pooled R2 than Graph4CFM (even with the full graph) in all three information settings.
We also evaluated the new Graphs4CFM checkpoint on CausalChambers. All methods receive 192 context rows: 192 observational rows for Graph4CFM and CIDER-FM-Obs, and 96 observational plus 96 surrogate-interventional rows for CIDER-FM.
CausalChambers: nine acquisition blocks, three context draws per block
CIDER-FM improves on Graph4CFM across the reported metrics in this real-world setting.
Accordingly, our principal CFM comparisons are CIDER-FM-Obs, which provides the matched architecture and training-budget control for the contribution of experimental context, and the newly trained Graph4CFM, which provides an external graph-conditioned CFM comparison under a matched natural prior. Tabular foundation models such as TabPFN serve as supplementary predictive baselines.
We added a new paragraph ("CFMs with experimental context") to related works (Section 2, pp. 3-4) where we explain that CIDER-FM is, to our knowledge, the first CFM that predicts conditional interventional distributions p(y∣do(t),x) from a context that combines observational data with data from several experimental regimes, including surrogate experiments on variables other than the treatment, in tabular systems with hidden confounding.
More specifically: